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Introduction 


The  purpose  of  this  IDEA  proposal  is  to  gather  preliminaiy  data  that  demonstrate  the  feasibility  of  visual 
servomg  optical  microscope  (VSOM)  technology,  experiments  and  assays.  We  hope  to  use  VSOM  in  future 
studies  to  rapidly  analyze  large  populations  of  living  cells  that  have  been  freshly  derived  from  human  breast 
cancer  tissue  biopsies.  This  could  lead  to  rapid  advances  in  in  vitro  chemosensitivity  testing  and  would  represent 
a  large  step  toward  our  ultimate  goal  of  chemotherapy  regimens  tailored  to  the  individual  breast  cancer  patient. 
We  believe  the  rapid  analysis  of  dynamic  physiological  responses  on  a  patient-by-patient,  tumor-by-tumor,  and 
cell-by-cell  basis  will  provide  critical  information  for  individualized  cancer  chemotherapy.  Such  assays  could 
produce  detailed  and  distinct  physiological  fingerprints  that  are  highly  predictive  of  an  individual  patient's 
response  (both  normal  and  tumor  cell  response)  to  a  particular  anti-cancer  drug,  particular  combination  of  drugs, 
or  particular  sequence  of  drug  exposures. 

Visual  servoing  (VS)  is  a  term  from  the  field  of  robotic  vision,  and  it  refers  to  the  dynamic  manipulation  of 
experimental  parameters  based  on  analysis  of  digital  image  content.  A  VSOM  assay  consists  of  (i)  real-time 
observations  of  individual  cell  responses  to  a  series  of  applied  stimuli  (ii)  adaptive,  and  intelligent  adjustment  of 
experimental  parameters  to  achieve  experimental  goals.  In  VSOM  assays,  the  flow  rates  of  computer-controlled 
syringes  are  adjusted  based  on  observation  and  quantification  of  cell  responses  in  a  cell  chamber  (see  Era  of 
Hope  Poster,  Appendix  C).  On-line  image  segmentation  and  analysis  of  many  single  cell  responses  is  required 
for  VSOM  experiments,  with  subsequent  software  control  of  the  syringes.  We  have  met  these  requirements  in 
experiments  where  we  monitor  up  to  1000  living  cells.  In  Year  2,  we  believe  we  have  demonstrated  the 
feasibility  of  a  new  class  of  in  vitro  drug  response  and  chemosensitivity  assays;  namely,  VSOM  assays  that  are 
based  on  a  series  of  quantified,  dynamic  physiological  responses  of  intelligently  perturbed  single  cells. 

An  ideal  VSOM  chemosensitivity  assay  does  not  irreversibly  perturb  living  cells,  and  it  allows  one  to  make 
predictions  of  drug  sensitivity  based  on  specific  physiological  characteristics  of  individual  cells.  We  believe  our 
technology  will  allow  us  to  rapidly  (i)  develop  new  in  vitro  fluorescence  assays  that  can  be  used  to  predict 
chemosensitivity  (ii)  develop  selective  culture  conditions  for  specific  subpopulations  of  cells,  i.e,  primary  culture 
tumor  cells  (iii)  demonstrate  a  new  paradigm  in  in  vitro  chemosensitivity  testing.  This  new  paradigm  is  visual 
servoing  for  knowledge-based  interrogation  of  individual  living  cancer  cells. 

Several  aspects  of  our  approach  are  novel.  We  have  combined  sophisticated  robotic  vision  techniques,  digital 
video  microscopy  and  biophysics  to  monitor  and  automatically  respond  to  the  responses  of  hundreds  of  living 
cells.  We  have  successfully  demonstrated  VSOM  technology  by  analyzing  the  heterogeneity  and  physiological 
characteristics  of  mked  populations  of  drug-sensitive  (DS)  and  multidrug  resistant  (MDR)  human  breast  cancer 
cells  in  real-time.  This  demonstrates  the  advantages  of  a  VSOM  doxorubicin  chemosensitivity  assay  based  on  the 
dynamic  physiological  responses  of  single  cells.  These  results  indicate  our  approach  will  be  extremely  useful  for 
developing  new  VSOM  chemosensitivity  fluorescence  assays  for  a  variety  of  breast  cancer  drugs.  In  addition,  we 
have  begun  to  develop  assays  for  cell  proliferation  in  living  cells,  which,  together  with  assays  for  apoptosis,  will 
allow  us  automatically  and  intelligently  design  the  appropriate  microenvironmental  conditions  for  the  propagation 
of  primary  human  breast  tumor  cells. 
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Body 


The  funding  for  our  original  3-year  IDEA  proposal  was  cut  33%,  and  the  time  for  completion  was  reduced  to  2 
years.  However,  we  requested  and  obtained  a  no-cost  extension  so  that  we  may  complete  and  publish  several 
critical  aspects  of  this  research  in  Year  3,  as  originally  proposed.  Thus,  we  present  the  results  of  Year  2  in  this 
Aimual  Report,  and  will  submit  a  Final  Report  at  the  end  of  Year  3.  Our  Technical  Objectives  (as  stated  in  the 
Two  Year  Revised  Statement  of  Work)  were: 

1)  Technical  Objective  1  (Tasks  1-5,  System  Development)  :  Develop  an  automated  VSOM  system. 

2)  Technical  Objective  2  (Tasks  6  -10,  Biological  Experiments);  (a)  identify  subpopulations  of  MDR  cells  in 

mixtures  of  DS  and  MDR  human  breast  cancer  cells,  (b)  characterize  differences  in  the  physiology  of  DS  and 

MDR  human  breast  cancer  cells,  (c)  identify  and  characterize  physiological  differences  between  normal, 

immortally  transformed,  and  tumorigenically  transformed  human  mammary  epithelial  cells  (HMECs). 

Another  important  goal  was  to  use  preliminary  data  to  obtain  new  funding,  and  we  have  successfully  obtained 
new  DOE  Medical  Applications  Program  funding  for  this  research  (included  in  Appendix  G:  Automated 
Imaging  System  for  Guiding  Antisense  Compounds  to  Specific  mRNA  targets  in  Living  Cells,  74655,  Office  of 
Biological  and  Environmental  Research  (OBER)  of  the  Office  of  Science  (OS),  U.S.  Department  of  Energy 
(DOE)).  In  addition,  we  have  applied  for  a  DoD  Clinical  Bridge  Grant  so  that  we  can  bring  this  innovative 
technology  and  a  new  approach  to  chemosensitivity  testing  into  the  clinic  where  it  could  improve  clinical 
outcomes  for  breast  cancer  patients  (included  in  Appendix  G;  Rapid  Discovery  of  Physiological  Characteristics 
Which  Distinguish  Malignant  and  Nonmalignant  Human  Breast  Epithelial  Cells,  BC000508,  Breast  Cancer 
Research  Program,  Congressionally  Directed  Medical  Research  Program  2000).  As  discussed  in  this  grant 
proposal,  we  are  now  able  to  integrate  a  Cancer  Biologist  into  our  effort,  in  a  way  that  directly  addresses  problems 
with  primary  breast  cancer  and  metastatic  cell  culture  techniques.  In  this  grant,  we  also  address  the  basic 
limitations  of  cell-line  in  vitro  chemosensitivity  and  predictive  assays,  and  we  propose  a  detailed  3 -year  strategy 
for  bringing  this  technology  into  clinical  trials. 

Technical  Objective  1  (Tasks  1-5,  System  Development) 

Task  1;  Develop  a  graphical  user  interface  (GUI)  for  a  prototype,  non-automated  VSOM  system. 

It  was  necessary  to  break  the  VSOM  GUI  development  into  two  separate  projects:  one  for  on-line  experiment 
control  and  data  acquisition,  and  a  separate  one  for  off-line  data  reduction,  annotation,  archiving,  and 
visualization.  As  shown  in  Figure  1  (Appendix  B),  we  have  created  a  JAVA-based  GUI  for  the  latter  project. 
This  program  is  referred  to  as  a  Java  Command  File  Generator  (JCFG),  and  it  is  currently  being  used  to  aggregate 
annotation  data  from  different  sources  (user  input,  laboratory  notebooks,  local  MS  Access  databases,  ICS  image 
header,  pump  logs,  recipe  files,  etc.)  into  a  label-based,  ASCII  text  command  file  that  can  easily  be  converted  to 
an  XML  format.  This  command  file  is  currently  accessed  by  a  variety  of  PC-based  data  reduction  and 
visualization  programs.  It  is  also  accessed  by  our  current  UNIX-based  Image  Library  (ImgLib)  software,  and 
various  (local,  in-house)  image  processing  routines  written  using  a  custom,  compiled  C  version  of  Scil  Image 
(TNO  Institute  of  Applied  Physics,  Delft,  The  Netherlands).  Various  PC-based  image  processing  (Corel 
PhotoPaint),  video  editing  (Ulead  Media  Studio  Pro),  and  plotting  packages  (Sigma  Plot,  SPSS  Inc.,  Chicago,  IL) 
equipped  with  Visual  Basic  automation  features  also  access  these  portable  text  files.  The  JCFG  was  produced 
using  Visual  Cafe,  Database  Edition  (Symantec).  Obviously,  our  current  level  of  funding  does  not  allow  us  to 
build  all  the  capabilities  found  in  PC  software  into  a  VSOM  GUI  at  the  present  time.  However,  we  have  adopted 
a  common  approach  of  building  platform  independent  Java-based  GUIs  that  can  aggregate  annotation  data 
collected  from  separate  locations.  This  allows  the  Biology  and  Biophysics  group  (Drs.  Callahan,  Yaswen, 
Dairkee)  to  use  PC  and  Macintosh  host  computers  in  addition  to  the  UNIX-based  Sparcstations  used  by  the 
Computer  Science  and  Bioinformatics  group  (Dr.  Parvin).  In  addition,  user  interfaces  that  have  been  previously 
used  and  tested  by  the  Biology  and  Biophysics  group  can  be  easily  integrated  Dr.  Parvin’ s  more  sophisticated 
Bioinformatics  System  (DeepView),  if  they  have  been  developed  in  Java.  We  have  recently  received  funding 
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{Automated  Imaging  System  for  Guiding  Antisense  Compounds  to  Specific  mRNA  targets  in  Living  Cells  )  that 
will  support  this  integration  and  development  work  with  Dr.  Parvin  and  Deep  View. 


We  are  currently  developing  visualization  and  interactive  data  features  that  include  the  ability  to  retrieve 
image  data  from  the  storage  system,  overlay  the  cell  number  label  and  the  final  cell  outline  produced  by  VSOM 
algorithms  on  any  combination  of  image  channels,  plot  time-dependent  cell  responses,  and  display  cell  responses 
as  time-lapse  digital  video.  As  we  demonstrate  throughout  this  report,  it  is  difficult  to  visualize  and  interact  with 
the  large  number  of  cell  responses  that  are  collected  across  multiple  channels,  even  when  the  analysis  is 
performed  off-line  with  color-coded,  3D  graphs  and  time-lapse,  enhanced  digital  video.  One  approach  we  are 
taking  is  testing  and  developing  analysis  and  visualization  of  our  data  with  currently  available  PC-based  scientific 
software  packages,  and  then  building  similar  functionality  into  Deep  View.  The  current  Java  VSOM  Results 
Browser  (Figure  2)  is  an  example  of  this  approach.  Data  from  experiment  "WAS"  is  shown.  (We  will  present 
data  from  this  particular  experiment,  WAS,  using  a  variety  of  visualization  and  analysis  tools  throughout  this 
report).  The  details  of  experiment  WAS  are  presented  later  in  this  report. 

The  following  is  an  overview  of  VSOM  system  integration  and  algorithm  development  from  a  Computer 
Science  perspective.  It  should  be  noted  that  several  Computer  Science  and  Bioinformatic  projects  supported  the 
development  of  tools  and  infrastructure  that  are  now  available  to  our  VSOM  system.  The  basic  VSOM  target 
instrument  is  an  inverted  optical  microscope.  We  have  developed  a  plugin  that  interfaces  this  instrument  to 
DeepView  (See  http:/Aision.lbl.gov/).  DeepView  is  a  channel  for  distributed  microscopy  and  informatics,  and  it 
is  funded  through  the  DOE  National  Collaboratoty  program.  DeepView  provides  a  scalable  interface  for  adding 
any  instrument  into  its  framework.  It  also  provides  the  means  for  data  transfer  and  viewing  over  the  wide  area 
network.  The  system  has  a  generic  graphical  user  interface  that  interacts  with  the  target  instrument  through 
advertisement  of  its  properties.  In  the  case  of  VSOM,  related  properties  are  computer  controlled  syringes, 
shutters,  an  XY  stage,  a  Z-axis  focus  motor,  a  filter  wheel,  and  camera  control  parameters.  This  concept  of 
advertising  capabilities  (functionalities)  of  an  instrument  leads  to  a  more  uniform  interface  and  reduces 
maintenance  cost  through  software  reusability.  One  way  to  achieve  this  is  through  implementing  a  plugin,  which 
has  been  accomplished  for  VSOM. 

The  current  functional  architecture  for  visual  servoing  is  shown  in  Figure  3A.  It  consists  of  an  image- 
acquisition  module,  image-analysis  module,  servoing,  and  archive.  The  image-acquisition  module  provides  the 
means  for  time-lapse  high  resolution  video  microscopy  with  six  fluorescence  excitation  filters.  This  module  has  a 
recipe  manager  to  allow  either  manual  or  preprogrammed  capture  of  images  at  different  temporal  and  excitation 
frequencies.  This  recipe  is  expressed  in  XML  notation  that  provides  semantic  interoperability.  The  analytical 
module  integrates  a  unique  segmentation  algorithm  that  provides  a  feature-based  summary  of  images  based  on 
size,  location,  response,  and  the  bounding  contours.  The  servoing  module  controls  the  concentration  of  various 
compounds  in  the  cell  chamber  by  controlling  the  flow  in  any  of  the  four  syringes  that  sit  next  to  the  microscope. 
The  servoing  provides  three  operational  modes  with  each  mode  subsequently  registered  with  the  recipe-manager: 
(1)  a  static  recipe  in  which  the  flow  rate  of  each  syringe,  its  start  point  in  time,  and  its  duration  is  specified;  (2)  a 
dynamic  recipe  in  which  the  flow  rate  and  its  duration  are  altered  as  a  function  of  a  particular  response  of  the  cells 
under  observation;  and  (3)  a  modulating  recipe  under  program  control  that  turns  the  flow  on  and  off  at  a  specified 
frequency.  The  archival  system  stores  the  images,  the  computed  responses,  and  annotation  data  in  a  flat  file 
environment. 

The  user  can  enter  a  recipe  for  a  particular  experiment.  The  recipe  allows  for  (a)  controlling  the  perfusion 
rate(s)  into  the  cell  chamber,  (b)  setting  camera  exposure  times,  (c)  selecting  the  location  of  the  optical  filter 
wheel,  and  (d)  setting  the  sampling  rate  for  data  collection.  There  are  currently  two  operational  VSOM  software 
packages  for  image  acquisition  and  control.  These  are  called  "VX5"  and  "ADR".  VX5  is  for  static  recipe  control 
of  the  system,  while  ADR  can  run  dynamic  recipe  visual-servoing  experiments  locally  or  over  the  network.  In 
Figure  3C,  a  sample  XML  recipe  file  for  a  VX5  run  is  shovra,  together  with  an  example  of  data  found  in  the 
pump  log  (Figure  3B),  and  an  example  of  an  ICS  digital  image  header  (Figure  3D). 

It  should  be  noted  that  image  and  experimental  annotation  features  are  being  developed  simultaneously  with 
new  VSOM  imaging  protocols.  For  example,  our  preliminary  results  have  indicated  that  it  is  necessary  to  return 
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repeatedly  to  the  same  set  of  cells  after  the  dish  has  been  returned  to  the  incubator  for  24-48  hrs.  In  order  to 
successfiilly  predict  the  ultimate  biological  endpoints  induced  by  the  stimuli  we  apply  and  the  physiological 
responses  we  observe,  we  require  this  functionality.  One  approach  currently  under  development  is  shown 
schematically  in  Figure  4,  where  a  cell  response  is  logged  on  the  basis  of  the  cell’s  2D  location  in  the  dish.  Using 
adaptive  visual  servoing  principles  for  XY  stage  control  ,  one  can  then  return  the  dish  to  the  microscope  stage 
days  later  and  return  to  the  same  field  of  cells.  Cells  that  have  disappeared  (perhaps  via  apoptotic  digestion),  lost 
viability,  moved,  or  divided  can  then  be  tallied.  As  seen  schematically  in  Figure  4,  a  fiduciary  mark  “+”  is  etched 
in  a  plastic,  35  mm  cell  culture  dish  under  sterile  conditions.  The  (0,0)  mark  is  recorded  by  the  VX5  program, 
which  then  keeps  track  of  the  relative  x,y  position  of  the  pertinent  camera  field  of  view.  A  detailed  protocol  for 
acquiring  VSOM  data,  and  then  repeatedly  returning  to  the  same  field  of  view  is  presented  in  Appendix  E. 


Task  2;  Optimize  algorithms  for  detecting  individual  cells  in  transmitted  light  by  analyzing  image  data  off-line. 

Recently,  we  have  developed  three  new  techniques.  One  is  based  on  a  new  regularized  edge  detector  based  on 
supremum,  the  second  on  a  vector  representation  of  the  underlying  image  followed  by  feature  extraction  in  vector 
space,  and  the  third  on  a  geometric  and  texture  model  of  a  cell  and  its  interaction  with  neighboring  cells.  The  new 
regularized  edge  detector  has  its  root  in  visualization  of  3D  objects;  however,  we  have  also  found  it  to  be 
applicable  to  segmentation  of  scientific  images.  The  vector  representation  of  images  has  its  root  in  analysis  of 
spatio-temporal  satellite  images.  Current  segmentation  results  for  living  cells  are  shown  in  Figure  5A-F.  For  an 
introduction  and  overview  of  fluorescence  microscope  digital  imaging,  see  the  Era  of  Hope  Poster  in  Appendix 
C. 

A  regularized  edge  detector  based  on  supremum;  We  have  developed  a  technique  that  accurately  delineates 
target  nuclei  and  their  boundaries  from  background.  This  technique  (a  regularized  edge  detector  based  on 
supremum)  is  superior  to  our  previous  method  that  relied  on  a  parametric  description  of  target  nuclei  in  the  form 
of  a  projection  of  a  sphere,  better  known  as  hough  transform.  Current  regularization  techniques  attempt  to 

minimize  an  integral  such  as  ^  |y/'|^ .  Yet,  this  formulation  has  no  control  on  the  local  property  of  / .  In  other 
words,  the  global  average  of  |y/’j  may  be  small,  but  locally  /may  change  sharply.  A  way  to  overcome  this  issue 
is  to  minimize  |y/'|  at  every  point,  which  leads  to  minimization  of  the  supremum  of  jV/j .  It  is  natural  to  consider 


the  functional  H  ^  (/)  =  5«Pr|v/|  as  a  "limit"  of  the  sequence  of  functionals 

H„(/)=  (  ||V/|“<fe  )2';v  ,  N  =  1.2,3,...  (1) 

The  Euler  equation  for  the  minimization  of  the  functional  H  ^  (/)  can  be  expressed  as: 

V/  +  /,V„  +  2/,/,/„  +  fif„  j  =  0  (2) 

df  f 

where  a  subscript  indicates  a  derivative,  such  as  =  — ,  f  = - .  By  removing  the  first  coefficient  and 

dx  ^dy 

letting  iV  ->  00 ,  we  have 

///=+2A4A,  +  /,%  =  o  (3) 


Equation  (3)  is  called  the  Infinite  Laplacian  Equation  (ILE),  which  has  been  studied  widely  in  the  literature. 
Some  important  properties  of  the  equation  (2)  are:  (1)  there  is  at  most  one  solution  and  if  we  redefine  the 


Annual  Report:  Callahan,  Parvin  Page  7 


"solution"  in  a  suitable  weak  sense,  then  a  continuous  solution  to  C'  does  exist;  (2)  the  trajectory  of  the  gradient 
of  f  is  either  a  convex  curve  or  a  straight  line;  and  (3)  there  are  no  stationary  points  |V/|  =  0  in  R  . 

The  rationale  for  optimality  of  the  supremum  is  based  on  the  "equal  importance  criterion".  This  criterion 
asserts  that  every  point  in  R  is  equally  important  and  contributes  similarly  to  the  estimation  process.  Any  other 
assumption  means  that  we  need  to  know  some  additional  information  about  the  curve.  It  is  easy  to  verify  that  ILE 
is  equivalent  to  the  follow  equation: 


3'(/)=v(|v/|).i^  =  0 


which  implies  that  along  each  trajectory  of  the  gradient  of  f ,  the  magnitude  of  the  gradient  is  a  constant. 
Although  the  overall  integral  of  |y/’[  may  be  larger,  the  supremum  is  smaller  and  the  gradient  is  more  likely  to 
concentrate  at  a  smaller  range.  It  is  interesting  to  note  that  Equation  3  is  equivalent  to  the  zero-crossing 
{fxx  fyy  the  local  coordinate  system,  where  the  local  coordinate  system  is  defined  by  Wf .  The  edge 

detection  results  reveal  closed  regions.  These  regions  correspond  to  spatial  blobs  that  are  further  filtered  for  size, 
response,  and  convexity  constraint.  An  example  is  show  in  Figure  5A3- 

Tensor-based  feature  analysis:  Vector  field  analysis  is  a  well-studied  technique  in  computer  vision  and 
pattern  recognition  as  well  as  other  classic  fields  such  as  mathematics  and  physics.  Many  concepts  and  tools  have 
been  developed  for  motion  based  applications.  Our  work  extends  the  current  state  of  the  art  to  segmentation  and 
interpretation  of  scientific  images.  Given  an  intensity  image  there  exists  a  natural  vector  field  derived 

from  the  intensity  image,  which  corresponds  to  the  gradient  field  rotated  by  : 


The  problem  here  is  that  the  vector  field  v  is  noisy,  and  some  type  of  regularization  needs  to  be  introduced.  This 
can  be  expressed  as 

min  fflju  -  v||^  -i-  |Vu|j^  dxdy  (b) 


where  u  is  the  regularized  vector  field.  An  elliptic  PDE  can  be  solved  iteratively.  A  first  order  approximation 
can  be  computed  by  a  simple  linear  (Gaussian)  scale  space  model: 

/(■;<)=«(■;<)•/.(•)  (7) 

where 


is  the  Gaussian  kernel  with  standard  deviation  <j  =  . 


Singularities  of  the  vector  field  then  provide  a  compact  abstraction  for  the  dense  vector  representation.  These 
singularities  can  be  characterized  by  point  or  linear  features.  Point  features  include  vortices.  An  example  of  such 
a  vector  field  and  its  singularities  (vortices)  on  a  synthetic  scene  is  shown  in  on  Page  10  (A-D)  of  the  Era  of 
Hope  Poster  (Appendix  C).  On  page  10  of  the  poster,  vector  field  analysis  on  a  synthetic  image  is  shown  in 
four  steps:  (A)  original  image;  (B)  vector  field  corresponding  to  the  gradient  field;  (C)  detection  of  vortices  from 
the  vector  field;  (D)  localization  results.  Once  these  patterns  are  detected,  the  corresponding  objects  can  be 
extracted.  Rao  used  local  Jacobian  for  feature  extraction  from  the  underlying  vector  field  to  detect  singularities. 
Our  approach  is  more  robust  and  provides  a  measure  for  the  size  of  the  vortex  as  well.  The  vortex  size 
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complements  localization  of  convex  blobs  even  though  they  are  connected  (or  touching  one  another).  This  is 
based  on  Jordan  index  to  localize  singularities  in  the  underlying  vector  field.  Let  F  =  (u,  v)  be  a  vector  field  and 
J  be  a  Jordan  curve  with  no  critical  point  on  it.  The  index  of  J  is  defined  by 


2  2 
U  -h  V 


At  each  point  P ,  we  choose  a  small  circle  of  radius  R  (denoted  by  Jp  )  around  P  and  compute  Index ). 
The  flow  field  («,  v)  can  then  be  classified  according  to: 


1 .  The  index  of  a  vortex  is  equal  to  +1  (the  classification  of  singular  points  in  a  vector  field  has  been  given  in  the 
literature). 


Note  that  there  is  no  node  in  the  vector  field  because  its  divergence  is  zero  everywhere,  i.e.. 


div\ 


^  V  ^  J  ^  5)C 


(10) 


The  vortex  size  is  then  estimated  by  a  simple  search  technique.  If  a  point  (jc,  y)  is  a  vortex,  then  its  size  i?*  (x,  y) 
can  be  defined  as 


/?*(x,y)  =  max{/?  |  Index(j(“  y))=  l}  (n) 

In  other  words,  ./?*(x,y)  is  the  largest  R  such  that  the  index  of  remains  to  be  1.  Being  an  integral  of  the 

first  order  partial  derivatives,  the  Jordan  index  is,  in  some  sense,  a  fimction  of  the  intensity  image.  In  contrast, 
other  techniques  are  based  on  high  order  derivatives,  which  are  bound  to  be  more  noisy.  Page  10  (A-D)  of  the 
Era  of  Hope  Poster  (Appendix  C)  show  the  result  of  vortices  and  region  segmentation  on  a  synthetic  image. 
Figures  5C,D  show  results  on  the  nuclei  of  living  and  lysed  cells  labeled  with  the  fluorescent  dye  Hoechst  33342. 
Figures  5E,F  show  results  on  livii^  cells  imaged  using  a  20X  phase  contrast  object  and  transmitted  light.  The 
later  result  is  significant  because  current  techniques  for  segmenting  individual  cells  require  fluorescence  imaging 
which  adds  a  layer  of  complexity  to  the  design  of  experiments  involving  living  cfills 

NOTE;  Figures  5E,F  are  the  only  digital  images  presented  in  this  report  that  were  not  acquired  using  a 
lOX,  NA  0.5  Zeiss  Fluar  Objective.  All  images  are  1024x1024,  and  the  CCD  chip  in  our  camera  has  pixels 
of  the  size  6.8x6.8  p.  Thus,  all  images  taken  at  lOX  show  a  field  of  view  that  is  approximately  700p  high 
and  700p  wide. 

Geometric  and  texture  model:  Although  the  previous  techniques  are  applicable  for  localizing  nuclei,  a  more 
advanced  technique  is  needed  for  a  more  detailed  segmentation  at  higher  imaging  resnlntinns  This  is  based  on  a 
geometric  model  of  the  cell,  its  texture  in  various  compartments,  and  the  relationship  of  its  features  with  respect 
to  one  another.  The  geometric  model  is  expressed  as  a  graph,  and  the  texture  is  computed  using  an  array  of 
orientation  and  scale  specific  filters.  These  local  feature  activities  help  collect  an  ensemble  of  information  that  is 
further  refined  with  dynamic  programming.  Our  preliminary  result  shown  on  Page  10  fE)  of  the  Era  of  Hope 
Poster  (Appendix  C),  indicates  reasonable  s^mentation  of  nucleus  and  cytoplasm,  which  can  be  used  as  a  basis 
for  computing  responses  in  each  of  these  compartments.  Furthermore,  responses  can  be  annnt^tH  with  a 
proximity  filter  (distance  transform)  to  reveal  whether  responses  in  the  cytoplasm  have  a  preferred  localization  or 
are  difiused. 

Tasks  3-5;  Demonstrate  visual  servoing  using  computer-controlled  syringes  and  a  DS/MDR  model  cell  system. 
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A  matched  pair  of  drug-sensitive  (DS)  and  multidrug  resistant  (MDR)  cell  lines  were  used.  Both  the  DS  cell 
line  (MCF-7  WTC)  and  the  MDR  cell  line  (MCF-7  ADR)  were  obtained  from  Dr.  Amadeo  Parissenti  (North 
Eastern  Ontario  Regional  Cancer  Centre  (NEORCC),  Sudbury,  ONT,  Canada).  Both  our  lab  and  Dr.  Parissenti 
tested  these  cell  lines  using  a  B-test  kit  for  a  MultiDnig  Resistance  (MDR)  Assay  being  developed  by 
Molecular  Probes  (Eugene,  OR).  As  discussed  in  our  Year  I  Annual  Report,  a  similar  set  of  matched  MCF-7  cell 
lines  obtained  from  an  LBNL  colleague  (Dr.  Ruth  Lupu)  did  not  demonstrate  differential  accumulation  of  calcein 
or  Lysotracker  Red  (LTR).  Dr.  Parissenti  obtained  his  cell  lines  directly  from  the  published  originator  (Dr. 
Kenneth  Cowan),  and  verified  differential  calcein  accumulation  via  two  distinct  MDR  mechanisms  using  the 
same  MDR  Assay  P-test  kit  that  Molecular  Probes  had  provided  to  us.  The  p-test  kit  is  being  developed  for  use 
with  a  flow  cytometer,  and  Dr.  Parissenti  performed  his  verification  tests  using  flow  cytometric  methods.  This 
provides  us  with  an  opportunity  to  compare  VSOM  results  with  flow  cytometric  results.  In  addition,  we  have 
compared  VSOM  results  with  results  obtained  using  a  fluorescence  multiwell  plate  reader  (Cytofluar  4000).  The 
choice  of  this  assay  is  based  on  its  ability  to  demonstrate  the  abilities  of  the  VSOM  system  with  this  set  of  cell 
lines.  It  is  not  necessarily  the  best  protocol  or  assay  for  rapidly  and  accurately  determining  the  relative  amounts 
of  Pgp  and  MRP  molecules  per  cell. 

VSOM,  Cytofluar  4000,  and  Digital  Imaging  Tests  of  Molecular  Probes  MDR  Assay  P-Test  Kit: 

Principles  of  the  kit:  The  acetoxymethyl  ester  (AM)  derivative  of  the  intensely  fluorescent  dye,  calcein 
(CAM),  is  a  relatively  nonfluorescent,  nonpolar,  lipophilic  molecule  that  can  normally  diffuse  passively  across 
the  plasma  membrane.  Once  inside  the  cell,  the  AM  group  is  cleaved  from  CAM  by  intracellular  esterases.  The 
resulting  molecule,  calcein,  is  negatively  charged,  hydrophilic,  and  is  normally  retained  within  the  cell.  However, 
most  cells  have  the  ability  to  produce  on  demand  (overexpress),  a  variety  of  transmembrane  proteins  that  are 
energy  (ATP)  -dependent  transporters,  or  pumps.  Once  one  or  more  of  these  proteins  is  overexpressed,  the  cell  is 
said  to  have  acquired  the  multidrug  resistance  (MDR)  phenotype.  Interestingly,  any  given  MDR  protein  has  a 
broad  spectrum  of  substrates  (including  many  anticancer  drugs  and  fluorescent  dyes)  that  can  be  actively 
extruded,  thus  protecting  the  cell  from  possible  cytotoxic  effects.  Depending  on  the  MDR  molecule,  the  substrate 
may  be  intercepted  while  passing  through  the  plasma  membrane,  or  the  substrate  may  be  internalized  before  it  is 
pumped  out.  Substrates  that  can  be  extruded  by  a  given  MDR  protein  need  not  be  related  structurally,  or 
functionally.  The  model  cell  system  we  have  chosen  for  these  MDR  studies  is  the  MCF-7  Wild  Type  Cowan 
(MCF-7  WTC)  and  the  MCF-7  Adriamycin  (doxorubicin)  -resistant  line  MCF-7  ADR.  MCF-7  ADR  cells  are 
known  to  express  both  the  170  kDa  P-glycoprotein  (I^P)  and  the  190  kDa  Multidrug  Resistance  Protein  (MRP). 
Separate  genes  code  for  these  two  different  proteins.  The  drug  cross-resistance  profiles,  or  spectra,  of  these  two 
proteins  overlap,  but  are  not  identical.  In  addition,  these  two  different  proteins  have  different  sensitivities  to 
various  inhibitors,  and  this  is  the  basis  of  the  assay  studied  here. 

Doxorubicin  (DOX,  the  anticancer  drug  Adriamycin)  is  a  fluorescent  molecule.  DOX  efflux  is  the  classical 
functional  assay  for  MDR,  but  it  suffers  from  low  sensitivity,  unless  UV  excitation  light  is  used.  We  have 
substituted  DOX  for  CAM  in  several  experiments,  and  have  little  problem  detecting  its  signal.  We  may  use  DOX 
in  place  of  CAM,  which  is  now  patented  for  MDR  assays  (one  must  buy  the  kit,  if  one  wants  to  use  CAM  to 
perform  MDR  assays).  However,  CAM  is  now  favored  by  most  researchers,  and  we  used  CAM  for  most  of  our 
studies  in  Year  2. 

Calcein  accumulates  intracellularly  at  a  rate  that  is  dependent  on  several  factors:  (i)  external  CAM 
concentration,  (ii)  temperature,  (iii)  number  and  type  of  MDR  pumps  in  the  cell  membrane  which  can  extrude 
CAM,  (iv)  the  general  metabolic  state  of  cell,  including  energy  charge  and  endogenous  esterase  activity,  (v) 
presence  or  absence  of  MDR  pump  inhibitors  such  as  MK-571  ("MK"),  vanadate,  cyclosporin  A,  probenecid,  or 
verapamil  ("V"),  (vi)  the  kinetics  and  mechanism  of  inhibition  (e.g.,  competitive  vs.  noncompetitive  inhibition, 
etc.). 

Experimental  Details:  A  schematic  for  VSOM  study  WA3  is  given  in  Figure  6A.  This  experiment 
consisted  of  two  VSOM  runs.  Run  #1  for  MCF-7  WTC  (DS)  cells,  and  Run  #2  for  MCF-7  ADR  (MDR)  cells. 
Cells  were  grown  in  35mm  cell  culture  dishes  (Nunc,A  )  in  DMEM  containing  10%  FBS,  and  1:100  pen/strep. 
Cells  were  allowed  to  plate  at  least  24  h  before  each  run.  Runs  took  place  sequentially,  on  the  same  day.  Cells 
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were  rinsed  with  "D-PBS  containing  high  glucose  and  sodium  pyruvate"  (DPBS_GP,  or  BUFF),  and  were  then 
incubated  for  1  h  in  DPBS  GP  containing  Hoechst  33342  (H42),  which  is  membrane  permeable  and  labels  the 
nuclei  of  living  cells.  Cells  were  then  placed  in  a  temperature-controlled  microperfiision  chamber  (Figure  6G, 
and  Era  of  Hope  Poster,  pages  3  and  4)  and  the  VSOM  run  was  begun.  For  both  runs,  an  initial  buffer 
perfusion  (BUFF)  was  used  to  wash  out  the  H42  incubation  solution,  followed  by  perfusion  of  a  CAM  (only) 
solution.  Figure  9  gives  detailed  information  on  the  syringes,  composition  of  perfusion  solutions,  flow  rates,  and 
perfusion  intervals  used  in  this  study.  Pages  3  and  4  in  the  Era  of  Hope  Poster  (Appendix  C)  contain  overview 
photos  of  the  VSOM  system.  Continuing  in  Figure  6A,  one  can  see  that  the  CAM  (only)  perfusion  is  followed  by 
a  BUFF  (buffer,  DPBS  GP)  perfusion.  Next,  a  CAM+MK  perfusion  is  performed,  followed  by  a  buffer 
perfusion.  The  final  exposure  is  to  a  CAM+V  solution,  followed  by  a  buffer  wash.  Total  time  elapsed  is  1 10  min, 
or  1 .8h  (6600  sec).  As  depicted  in  Figure  6A,  the  DS  cells  rapidly  accumulate  calcein  during  the  initial  CAM 
(only)  perfusion,  while  MDR  cells  accumulate  little  or  no  calcein  due  to  the  presence  of  both  the  Pgp  and  MRP 
transmembrane  proteins.  Both  Pgp  and  MRP  can  intercept  CAM  in  the  plasma  membrane,  before  it  reaches  the 
inside  of  the  cell,  where  it  is  digested  to  form  calcein.  Interestingly,  MRP  can  also  extrude  calcein  from  the 
interior  of  the  cell,  while  the  PgP  protein  cannot.  MK-571  is  a  specific  inhibitor  of  MRP;  thus,  during  the 
perfusion  of  CAM+MK,  MDR  cells  begin  to  accumulate  calcein,  while  DS  cells  are  not  greatly  perturbed. 
Verapamil  inhibits  both  Pgp  and  MRP;  thus,  when  CAM+V  is  perfused  MDR  cells  should  (theoretically) 
equilibrate  to  the  same  intracellular  calcein  concentration  as  DS  cells  during  this  final  perfusion.  By  comparing 
the  fluorescence  intensity  of  cells  after  CAM+MK  perfusion  to  the  fluorescence  intensity  after  CAM+V 
perfusion,  one  can  determine  the  relative  amounts  of  MRP  and  Pgp  pumps  on  a  per  cell  basis. 

One  important  fact,  not  noted  in  Figure  9,  is  that  all  CAM  solutions  (CAM,  CAM+MK,  CAM+V)  contain 
between  0.1  and  0.5%  DMSO.  Concentrated  stock  solutions  of  CAM,  MK,  and  V  are  prepared  in  DMSO, 
because  they  have  limited  solubility  in  water.  Small  aliquots  of  the  stock  solution(s)  are  then  added  to  DPBS_GP. 
The  scientific  staff  at  Molecular  Probes  have  informed  us  that  there  are  some  toxicity  issues  involving  DMSO, 
even  at  these  low  concentrations.  Furthermore,  these  toxicity  issues  can  be  a  function  of  the  particular  lot  of 
DMSO  used. 

Cvtofluar  Experiment  WAl:  As  mentioned  above,  we  have  also  attempted  to  implement  this  MDR  assay 
using  living  cells  in  24-well  plates.  This  assay  more  closely  follows  the  Molecular  Probes  protocol  suggested  for 
flow  cytometric  studies  because  cells  are  only  exposed  to  a  single  MDR  modulator  (MK  or  V).  However,  in 
addition  to  using  CAM  as  a  fluorescent  probe,  we  also  used  DOX  as  a  fluorescent  probe  (both  at  0.25pM)  in  a 
separate  set  of  plates.  On  Day  1,  DS  (MCF-7  WTC,  2  plates)  and  MDR  (MCF-7  ADR,  2  plates)  cells  were  plated 
in  24-well  plates  at  a  density  of  SxlO**  cells/well  or  2.8x10'*  cells/cm^.  In  addition,  a  concentration  ladder  of  wells 
was  generated  that  received  from  0  to  IxlO’  cells/well.  The  layout  for  all  four  plates  is  shown  in  Figure  7A.  An 
8-channel,  automated  electronic  pipettor  with  a  programmable  console  (Electrapette,  Matrix  Technologies, 
Lowell,  MA)  was  used  in  these  experiments  to  reduce  pipetting  errors.  Aliquots  were  drawn  from  a  Wheaton 
sterile  reservoir  ("V-shaped"  bottom)  (F^ure  6B).  Cells  were  then  grown  in  5%  CO2  in  DMEM  containing  10% 
FBS  and  Pen-Strp.  On  Day  3,  media  was  removed  and  wells  were  rinsed  with  DBPS  GP.  Solutions  of  CAM, 
CAM+MK,  or  CAM+V  and  solutions  of  DOX,  DOX+MK,  or  DOX+V  were  then  added  to  the  appropriate  wells 
of  the  appropriate  plates  for  a  Ih  exposure  at  37°C,  5%  CO2.  Control  wells  received  CAM  (only)  or  DOX  (only) 
solution.  After  a  Ih  exposure,  all  wells  were  rinsed  with  DPBS  GP  (containing  1:100  pen-strep),  and  plates 
containing  living,  exposed  cells  were  read  on  a  temperature-controlled  multichannel  fluorescence  plate  reader  at 
37°C  (Figure  6C).  After  this  first  set  of  measurements,  plates  were  returned  to  the  incubator  for  an  additional 
24h.  On  Day  5.  living  cells  were  triple  stained  with  H42,  CAM,  and  Ethidium  homodimer  (EH,  a  red  fluorescent 
indicator  for  the  nuclei  of  dead  cells),  giving  blue,  green,  and  red  fluorescence,  respectively,  on  separate  channels. 
After  this  second  set  of  measurements,  cells  were  fixed  and  on  Day  6,  cells  were  stained  with  DAPI  (a  DNA 
stain),  and  a  third  set  of  fluorescence  measurements  were  acquired. 

Experiment  WAl  was  designed  to  show  many  of  the  strengths  of  multiwell  plate  assays.  On  Day  3.  multiple 
readings  (of  whole  cell  populations)  were  taken  at  different  positions  within  each  well,  and  replicate  samples  were 
measured  in  adjacent  wells.  Thus,  the  calcein  accumulation  for  each  exposure  condition  was  easily  obtained. 
Readings  were  corrected  for  background  fluorescence  using  an  average  background  value  given  by  the  four  blank 
(no  cells)  wells.  All  four  plates  could  be  rapidly  read  (<5min  per  plate)  with  the  lids  on.  The  lids  can  be  left  on 
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because  fluorescence  excitation  occurs  from  the  bottom  and  measurements  are  taken  through  the  bottom  of  the 
plate.  All  plates  could  then  be  returned  to  the  incubator  for  several  days.  On  Day  5.  it  was  then  possible  to 
relabel  the  plates  using  three  different  fluorescence  dyes.  H42  was  used  to  label  cell  nuclei  (blue  channel),  thus 
giving  an  idea  of  total  cell  number  per  plate.  CAM  (only)  and  EH  gave  green  and  red  readings,  respectively,  that 
indicate  the  relative  number  of  live  and  dead  cells  (EH  only  stains  the  nuclei  of  non-viable  cells).  On  Day  6, 
fixed  cells  could  once  again  be  stained  with  DAPI  so  that  the  relative  number  of  cells  per  well  could  be  obtained 
for  purposes  of  normalizing  observed  fluorescence  to  relative  number  of  cells  present. 

Results  from  experiment  WAl  are  shown  below  in  Table  1. 


TABLE  1 

PLATE  EXPOSURE  DAY  3:  Signal  <490nm>  ±  STD  DAY  S:  Signal  f490nm>  ±  STD 

fall  Plates.  1.0  uM  CAM.  1  hr! 


WTC_CAM 

Control 

398  ±86 

4822  ±973 

(DS) 

MK-571 

553  ±24 

4951  ±427 

Verapamil 

567  ±558 

4997  ±599 

MDR_CAM 

Control 

305  ±61 

1658  ±36 

(MDR) 

MK-571 

211  ±175 

1588  ±158 

Verapamil 

164  ±134 

1373  ±178 

WTC_DOX 

Control 

n.o. 

6598  ±429 

(DS) 

MK-571 

n.o. 

6356  ±458 

Verapamil 

n.o. 

6556  ±733 

MDR_DOX 

Control 

n.o. 

1965  ±49 

(MDR) 

MK-571 

no. 

1641  ±400 

Verapamil 

n.o. 

1682  ±242 

Normalization  of  signal  in  the  490nm  channel  using  H42  (360nm  channel),  CAM/EH  ratio  (live/dead  ratio, 
490nm/530nm  channel),  or  DAPI  fluorescence(360nm  channel)  did  not  improve  standard  deviation  values  (data 
not  shown).  As  seen  in  the  Day  3  column  of  Table  1,  the  signal  observed  in  the  490  channel  was  weak  (blank 
wells  gave  a  reading  of  40  ±  1).  The  weak  fluorescence  exhibited  by  DOX  in  the  490  channel  was  not  observable 
(n.o.)  above  background.  These  values  represent  the  fluorescence  signal  of  calcein  or  DOX  after  1  h  exposures  at 
0.25  pM.  The  values  in  the  Day  5  column  represent  calcein  accumulation  in  cells  that  were  incubated  for  Ih  with 
1.00  pM.  Unfortunately,  in  no  instance  was  it  possible  to  observe  a  statistically  significant  difference  between 
MK  and  V  exposed  cells.  However,  (Day  5)  DS  cells  consistently  incorporated  more  calcein  than  MDR  cells 
when  the  four-fold  higher  CAM  incubation  was  used.  The  lower  CAM  concentration  (0.25  pM)  used  on  Day  3  is 
the  concentration  recommended  for  the  MK  and  V  concentrations  used  (10  pM  and  50  pM  ,  respectively),  and 
this  is  sufficient  signal  for  flow  cytometry  analysis.  The  signal  due  to  this  lower  concentration  is  also  easily 
detected  in  our  VSOM  experiments  (below).  Thus,  the  fluorescence  plate  reader  (Cytofluar  4000.  PE  Biosystems, 
Foster  City,  CA)  is  significantly  less  sensitive  than  VSOM  or  flow  cytometry.  However,  as  shown  in  F^ure  6F, 
multiwell  plates  can  be  easily  mounted  on  our  computer  controlled  scanning  stage.  VSOM  technology  could  be 
used  to  automatically  acquire  many  multichannel  images  in  each  well  of  interest.  It  is  interesting  to  note  some 
aspects  of  this  approach  (using  a  microscope  to  read  multiwell  plates)  have  been  recently  patented  {System  for 
cell-based  screening,  US5989835,  Inventor(s):  Dunlay;  R.  Terry ,  Pittsburgh,  PA,  Taylor;  D.  Lansing ,  Pittsburgh, 
PA.,  Applicant(s):  Cellomics,  Inc.,  Pittsburgh,  PA.,  Nov.  23,  1999.  See  www.cellomics.com,  and 
http://www.delphion.com/  for  more  information) 
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There  is  currently  great  commercial  interest  in  the  development  of  fluorescence  assays  for  microplate  readers 
(See  Cortese,  J.D.,  "At  the  Speed  of  Light"  The  Scientist,  ppl8-21,  July  10,  2000,  www.the-scientist.com).  A 
great  deal  of  progress  has  been  made  in  the  automation  of  these  systems  for  high-throughput  screening  (HTS). 
These  robotic  systems  are  fully  capable  of  performing  the  precisely  timed  solution  exchanges  and  multichannel 
fluorescence  detection  that  are  a  feature  of  VSOM  fluorescence  assays.  We  believe  one  important  application  of 
our  VSOM  technology  will  be  in  the  design  of  sophisticated  assays  that  be  implemented  by  these  high-throughput 
systems. 

Digital  ImagingTest  Experiment  WA2:  The  protocol  shown  in  Figure  6A,  and  used  in  experiment  WAl 
(Cytofluar  Test  of  MDR  Assay  P-Test  kit)  was  repeated  here  using  cells  plated  in  four  35mm  circular  cell  culture 
dishes.  DOX  was  not  used  in  this  study,  and  the  WAl  protocol  was  followed  only  up  to  Day  3,  at  which  point 
digital  images  where  obtained  from  dishes  (Figure  8,  Experiment  WA2).  In  this  color-coded  representation  of 
gray-scale  images,  low  intensity  pbcels  have  been  assigned  "colder"  colors  (black,  purple,  blue)  while  "warmer" 
colors  (yellow,  red,  white)  have  been  assigned  to  the  higher  intensity  pixels.  As  shown  in  the  bar,  the  warmer 
colors  indicate  higher  levels  of  intracellular  calcein.  MDR  (MCF-7  ADR)  incubated  with  CAM  (only)  are  shown 
in  Figure  8A.  These  cells  accumulate  relatively  little  calcein,  while  MDR  cells  incubated  with  CAM+MK 
(Figure  8B)  show  greater  accumulation,  and  MDR  cells  treated  with  CAM+V  show  the  highest  degree  of  calcein 
accumulation  (Figure  8C).  Figure  8D  is  an  image  of  DS  (MCF-7  WTC)  cells  stained  with  CAM  (only).  These 
digital  images  show  good  agreement  with  the  expected  behavior  of  the  P-Test  kit  assay  (F^ure  6A). 

VSOM  Experiment  WA3;  The  MDR  Assay  P-Test  kit  was  also  tested  in  a  VSOM  experiment  (WA3). 
Figure  9  is  a  schematic  diagram  of  this  VSOM  experiment,  which  is  also  depicted  in  Figure  6A.  Pumps  are 
turned  on  and  off  at  specified  flow  rates  according  to  a  pre-programmed  recipe,  or  based  on  real-time  analysis  of 
individual  cell  responses.  It  should  be  noted,  as  shown  in  Figure  9,  that  the  CAM,  V,  and  MK  concentrations  are 
the  same  as  those  used  in  experiments  WAl  and  WA2.  The  three  exposures  to  CAM  are  900s  (20nm))'each,  and 
thus  the  total  exposure  to  CAM  is  60min,  or  Ihr.  The  black  windows  at  the  bottom  of  the  figure  show  the  mean 
response  of  all  cells  in  the  field  of  view.  This  mean  response  plot  is  displayed  on  the  computer  screen  during  the 
VSOM  experiment,  and  digital  images  from  one  or  more  channels  (transmitted  light  is  usually  one  of  the  channels 
displayed)  are  displayed  on  the  screen  as  well.  The  plot  and  images  are  continuously  refreshed  and  updated 
during  the  experiment.  As  seen  in  the  two  plots  recorded  for  MCF-7  WTC  (DS,  run  #1)  and  MCF-7  ADR  (MDR, 
run  #2),  DS  cells  (upper  plot)  accumulated  more  calcein  (240±82,  N=336)  than  did  the  MDR  cells 
(103±80,N=161).  This  is  in  agreement  with  the  relative  differences  in  calcein  accumulation  observed  between  DS 
and  MDR  cells  in  the  multiwell  plate  experiments  (Table  1;  1658/4822  =0.34,  1965/6598=0.30;  VSOM:  103/240 
=  0.43)  which  represent  an  average  over  many  cells.  However,  as  shown  in  Figure  10,  (DS  cells.  Run  #1)  the 
mean  cellular  fluorescence  of  each  cell  is  monitored  during  VSOM  experiments,  and  deviations  from  the  mean 
can  be  easily  detected.  For  example,  the  arrow  in  Figure  10  indicates  several  cells  that  began  to  lose  viability  and 
plasma  membrane  integrity.  Such  cells  are  unable  to  retain  calcein,  and  it  leaks  out,  causing  the  fluorescence 
signal  to  drop.  It  is  possible  that  this  is  an  effect  related  to  the  DMSO  toxicity  discussed  above.  It  should  be 
noted  that  the  current  VSOM  system  could  easily  terminate  the  exposure/perfusion  as  soon  as  a  single  cell  death 
is  noted. 

Figure  11  shows  four  selected  single  cell  responses  from  Run  #2,  experiment  WA3.  These  response  curves 
are  good  examples  of  the  underlying  principle  of  the  P-Test  kit  shown  schematically  in  Figure  6A  (MDR,  RUN 
#2).  In  two  of  the  response  curves  (Figure  11,  red)  the  cells  do  not  accumulate  calcein  until  verapamil  (V) 
exposure  during  the  operation  of  Syringe  4  (Figure  9).  Thus,  we  can  infer  that  these  cells  only  express  Pgp 
pumps,  because  the  pumps  are  not  inhibited  by  MK-571  (MK).  It  can  clearly  be  seen  than  none  of  these  cells 
were  present  in  the  DS  cell  population  (F^ure  10).  Further,  after  CAM+V  exposure,  one  of  the  cells  rapidly 
accumulates  calcein  until  its  mean  cellular  fluorescence  falls  within  one  standard  deviation  of  the  mean  response 
of  cells  observed  for  DS  cells  (Figure  10).  If  we  assume  all  other  factors  are  the  same  then  one  would  predict  that 
the  second  cell  contains  fewer  Pgp  molecules.  The  remaining  two  curves  (Figure  11,  blue)  show  additional 
responses  during  CAM  (only)  and  CAM+MK  exposures,  requiring  a  additional  analysis.  Obviously,  the  actual 
situation  is  much  more  complicated  than  depicted  in  Figure  6A.  The  curves  we  observe  cannot  be  explained  in 
terms  of  the  presence  or  absence  of  only  two  pump  types,  neither  having  the  ability  to  extrude  internalized 
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calcein.  This  is  the  tvt?e  of  detailed  physiological  information  that  flow  cytometric  assays  and  multiwell  plate 
assays  cannot  provide,  and  it  demonstrates  that  our  VSOM  technology  has  the  capabilities  we  initially  predicted. 

For  example,  detailed  rate  information  on  calcein  accumulation  and  retention  is  available  throughout  the 
VSOM  experiment.  For  the  blue  response  curves,  one  can  calculate  a  rate  of  calcein  accumulation  during  the 
three  CAM  perfusions,  and  one  can  calculate  a  rate  of  calcein  retention  during  BUFF  perfusions.  Calcein 
accumulation  is  occurring  in  the  1500,  3500,  and  5500s  time  periods,  and  plateaus  corresponding  to  excellent 
calcein  retention  are  observed  in  the  2500,  4500,  6500  time  intervals.  As  noted  above,  some  MDR  pumps  have 
the  ability  to  extrude  calcein  after  it  has  been  internalized  in  the  cell.  The  ability  of  DS  cells  to  extrude 
internalized  calcein  is  apparent  during  the  2500s  period  (Figure  10)  corresponding  to  BUFF  perfusion;  however, 
2  MDR  cells  (Figure  11,  blue)  do  not  have  this  ability. 

Experiment  WA4:  We  have  also  performed  VSOM  experiments  where  only  CAM  (at  the  four-fold  higher 
concentration  of  1  .OpM)  is  perfused.  The  first  perfusion  interval  of  such  an  experiment  is  shown  in  Figure  12, 
where  the  same  data  is  presented  using  a  variety  of  3D  perspectives.  As  seen  earlier  (Figure  11,  blue),  a 
significant  subpopulation  of  calcein-accumulating  cells  is  present,  and  these  cells  demonstrate  varying  degrees  of 
calcein  accumulation  during  a  2100s  exposure.  In  this  experiment  CAM  solutions  were  not  flushed  from  the 
chamber  between  CAM,  CAM+MK,  and  CAM+V  exposures.  Cells  were  allowed  to  sit  in  contact  with  these 
solutions  between  perfusion  intervals.  Portions  of  this  experiment  are  presented  in  the  Biophysical  Society 
Abstract  (Appendix  D).  The  experiment  ended  after  1 1 00s  of  exposure(s),  and  although  the  cells  remained 
viable,  significant  membrane  blebbing  was  observed. 

Technical  Objective  2  ;  Biological  Experiments 

Tasks  6  -7;  In  our  original  three-year  proposal,  we  proposed  to  study  only  the  WTC  (DS)  and  MCF-7 

ADR  (MDR)  model  system  during  our  preliminary  VSOM  system  developn^iT  studies.  However,  it  was 
suggested  that  additional  cell  lines  might  give  more  pertinent  biological  information  and  might  yield  important 
biological  results  more  quickly.  Thus,  the  funding  was  reduced  33%,  and  the  number  of  cell  lines  to  be  studied 
was  doubled. 

As  we  discuss  in  our  Clinical  Bridge  Grant  Proposal  (Included  in  Provisional  Patent  application. 
Appendix  G:  Rapid  Discovery  of  Physiological  Characteristics  Which  Distinguish  Malignant  and  Nonmalignant 
Human  Breast  Epithelial  Cells,  BC000508,  Breast  Cancer  Research  Program,  Congressionally  Directed  Medical 
Research  Program  2000),  it  may  be  necessary  to  reproduce  the  low  pH  environment  seen  by  tumor  cells  in  vivo,  if 
we  are  to  successfully  culture  primary  breast  tumor  cells  in  vitro.  Our  objectives  are  to  demonstrate  and  verify 
that  VSOM  can  rapidly  discover  and  optimize,  on  a  tumor-by-tumor  basis,  microenvironmental  conditions 
suitable  for  the  successful  in  vitro  propagation  and  chemosensitivity  testing  of  primary  human  breast  tumor  cells. 
Our  specific  aims  are  to  (a)  rapidly  and  automatically  develop  in  vitro  microenvironments  that  favor  the 
propagation  of  primary  breast  tumor  cells,  (b)  rapidly  develop  new  in  vitro  fluorescence  assays  that  predict  cell 
behavior  on  the  basis  of  early  physiological  responses  to  applied  stress  (c)  improve  VSOM  technology  in  order  to 
achieve  our  objectives  and  specific  aims.  A  great  deal  of  progress  has  been  made  in  primary  culture  of  malignant 
human  breast  epithelial  cells  (BECs).  Currently,  there  is  great  interest  in  the  unique  physiology  of  solid  tumors, 
and  it  is  likely  that  a  sub  optimal,  nutritionally  depleted  environment  exists  in  breast  tumors.  This  environment  is 
believed  to  consist  of  regions  of  low  oxygen  (hypoxia),  low  pH,  low  levels  of  glucose,  and  high  levels  of 
metabolic  waste.  It  has  been  demonstrated  that  when  such  conditions  are  simulated  in  culture,  it  is  possible  to 
isolate  relatively  pure  populations  of  primary  breast  tumor  cells  .  Nonmalignant  cells  are  unable  to  survive  these 
initial,  hostile  environmental  conditions.  Thus,  an  additional  benefit  is  that  nonmalignant  epithelium  with  its 
higher  proliferation  rate,  is  not  present  and  is  not  able  to  overgrow  the  tumor  cells  which  have  a  slower 
proliferation  rate.  A  common  misconception  is  that  malignant  BECs  proliferate  more  rapidly  than  nonmalignant 
BECs.  It  should  also  be  noted  that  drug  resistance  in  tumor  cells  may  depend  on  and  be  the  result  of  the  stress  of 
a  hostile  microenvironment.  For  this  reason,  accurate  measurements  of  drug  resistance  may  require  that  the 
assays  be  performed  on  cells  in  the  proper  microenvironment.  It  is  interesting  to  note  that  some  of  the  most 
successful  in  vitro  chemosensitivity  tests  involve  the  culture  of  cells  in  tiny  capillary  tubes  that  are  sealed  at  both 
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ends  and  incubated  for  14  days  (see  below).  The  resulting  microenvironment  is  likely  to  be  even  more  hostile 
than  those  generated  using  the  sandwiched  coverslip  technique. 

We  immediately  encountered  pH-related  cell  propagation  effects  when  we  began  studies  on  human  mammaiy 
epithelial  cell  (HMEC)  lines.  Normal  HMECs  (184SK),  and  immortally  transformed  HMECs  (184B5)  provided 
by  our  LBNL  collaborator.  Dr.  Paul  Yaswen,  have  been  studied  We  have  successfully  conducted  VSOM  studies 
on  these  cell  lines.  However,  as  seen  in  Figure  13  (blue  circles),  MCF-7  WTC  and  MCF-7  ADR  cells  grown  in 
DMEM  (containing  a  bicarbonate  buffer)  and  HMECs  (Figure  13,  red  triangles)  grown  in  MEGM  (containing 
both  a  bicarbonate  buffer  and  a  HEPES  buffer)  show  very  different  pH  behavior  as  a  function  of  incubator  CO2 
levels.  In  addition,  the  pH  range  tolerated  by  each  cell  type  varies  (vertical  arrows  in  Figure  13).  The  net  effect 
is  that  there  is  not  a  common  CO2  level  that  allows  these  different  cells  types  (in  separate  flasks,  in  the  media  that 
each  prefers)  to  simultaneously  propagate  in  a  single  incubator.  The  more  normal  HMECs  tolerate  a  much 
narrower  range  of  pHs  than  do  the  MCF-7  cancer  cells.  None  of  the  cell  lines  prefer  a  lower  pH  environment 
(<7. 1 ).  If  it  is  true  that  successful  propagation  of  human  tumor  breast  epithelial  cells  requires  pHs  <  7.0,  then  it  is 
clear  why  previous  in  vitro  chemosensitivity  assays  have  not  been  successful.  Most  media  and  cell  incubation 
conditions  have  been  designed  for  pHs  >  7.0.  Thus,  when  human  tumor  biopsy  specimens  are  cultured  in  these 
conditions,  it  is  the  normal  cells  that  attach  and  proliferate,  not  the  tumor  cells.  Chemosensitivity  tests  are  then 
unknowingly  conducted  on  normal  cells,  rather  than  the  tumor  cells.  This  may  be  the  reason  for  the  previous 
failure  of  these  assays  during  clinical  trials. 

Figure  14  is  an  example  of  the  behavior  observed  when  an  attempt  is  made  to  propagate  cells  at  an 
unfavorable  pH.  In  this  case,  MCF-7  WTC  cells  incubated  at  CO2  levels  that  produce  a  pH  >7.4  (on  average)  in 
DMEM  successfully  attach,  spread  out,  and  proliferate  on  the  plastic  tissue  culture  ware  surface  (Figures  14 
A,B).  However,  at  pHs  <  7.2,  these  cells  round  up,  and  do  not  do  attach  well(Figures  14  A3)-  A  similar  effect 
is  seen  if  MCF-7  ADR  are  plated  under  conditions  where  the  pH  >  8.0.  In  this  case,  the  cells  never  successfully 
attach  or  proliferate,  while  MCF-7  WTC  cells  grow  well.  It  should  be  noted  that  these  type  of  morphological 
transformations  can  be  quantified  by  a  VSOM  system  with  the  ability  to  segment  cells  in  transmitted  light.  The 
images  in  Figure  14  were  acquired  by  placing  a  T-25  tissue  culture  flask  on  the  microscope  using  the  appropriate 
stage  mounting  plate  (Figure  6F).  In  the  future,  we  believe  the  VSOM  system  will  be  able  to  repeat^ly  image 
many  fields  of  cells  in  a  closed  tissue  flask.  The  flask  could  then  be  returned  to  the  incubator  for  a  suitable 
period.  Our  ability  to  segment  cells  in  transmitted  li^t  means  that  cell  shapes,  locations  and  relative  positions 
could  be  saved,  so  that  when  one  returned  the  flask  to  the  microscope,  the  VSOM  system  would  return  to  the 
same  fteld(s)  of  cells,  acquired  new  images,  and  quantify  any  morphological  changes.  This  is  an  example  of  a 
more  mechanical  visual  servoing  application,  where  the  system  would  relocate  the  same  field  of  cells,  even  if  cell 
numbers,  relative  positions,  and  shapes  had  changed. 

Tasks  8-9;  The  goal  of  these  tasks  was  to  find  and  exploit  differences  between  cell  types  to  give  a  growth 
advantage  to  specified  cells  (i.e.,  primary  breast  tumor  cells),  while  giving  a  growth  disadvantage  to  other  cells 
(i.e.,  normal  mammary  epithelial  cells).  In  addition  to  the  morphological  indications  (discussed  above)  that  cells 
have  attached  and  have  begun  dividing,  we  wish  to  develop  VSOM  fluorescence  assays  that  can  quantify  the 
proliferation  rate  of  living  cells  on  a  cell-by-cell  basis. 

Experiment  HMl:  BrdU  Proliferation  Assay  for  Lhii^  Cells 

The  power  of  any  VSOM  predictive  assay  is  improved  if  specific  cell  responses  can  be  correlated  with 
specific  biological  endpoints.  Several  fluorescence  assays  for  apoptosis  exist,  but  there  are  few  fluorescence 
assays  for  living  cells  that  can  measure  important  parameters,  such  as  BrdU  incorporation,  which  can  be  used  to 
quantify  DNA  synthesis  or  cell  proliferation.  In  experiment  HMl,  we  demonstrate  an  image  ratioing  technique 
that  can  detect  the  proliferation  state  of  living  cells  by  quantifying  the  amount  of  BrdU  incorporation.  The 
protocol  for  this  assay  is  illustrated  in  F^ure  7B,  and  is  adapted  from  the  published  protocol  used  by  our 
collaborator.  Dr.  Paul  Yaswen  (Stampfer,M.R.  et  al,  Exp.  Cell  Res.  208,  175-188  (1993).  We  used  the  same  cell 
lines  and  media;  however,  for  DNA  synthesis  assays,  we  substituted  5-bromo-2'-deoxy  uridine  (BrdU)  for  [^H]- 
thymidine.  We  performed  these  experiments  using  a  5-bromo-2'-deoxy  uridine  labeling  and  detection  kit  (No. 
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1296736,  Boehringer  Mannheim).  Thus,  after  a  VSOM  experiment  we  were  able  to  fix  the  cells  and  verify 
VSOM  results  (based  on  digital  image  ratioing)  using  traditional  indirect  immunofluorescence  assays. 

Principles  of  Assay:  Cells  that  have  been  stimulated  to  grow  (proliferate)  synthesize  twice  their  normal 
complement  of  DNA  in  preparation  for  cell  division.  These  actively  growing  cells  can  take  the  nucleotides  A,T, 
C,  or  G  from  the  extracellular  media  and  use  them  to  construct  DNA.  However,  when  BrdU  is  present,  it  is  used 
instead  of  T  to  synthesize  DNA.  Thus,  if  one  exposes  a  population  of  cells  to  BrdU  for  a  short  period  (1  h),  the 
subpopulation  of  cells  that  were  synthesizing  DNA  during  that  1  h  period  will  incorporate  the  most  BrdU  into 
their  nuclear  DNA.  This  is  referred  to  as  a  pulse-labeling  experiment.  The  goal  of  this  experiment  was  to 
determine  whether  the  ratio  of  Hoechst  33342  (H42)  and  Syto  16  (S16)  fluorescence  emission  intensities  is 
useful  for  determining  the  amount  of  BrdU  incorporation  on  a  cell  by  cell  basis.  H42  and  S16  stain  the  nuclei  of 
living  or  fixed  cells.  H42  fluorescence  emission  is  quenched  (reduced)  if  the  DNA  contains  incorporated  BrdU. 
The  fluorescence  emission  of  S16  is  not  affected  by  the  presence  of  incorporated  BrdU.  Thus,  the  ratio  of 
H42/S16  fluorescence  intensities  (calculated  on  a  pbcel  by  pixel  basis  using  digital  images)  should  be  proportional 
to  the  amount  of  BrdU  incorporated  into  each  cell’s  DNA. 

Three  dishes  of  184B5  cells  were  used  in  these  experiments  (Figure  7B).  For  the  first  48  hrs,  two  dishes 
received  MEBM  (Mammary  Epithelial  Cell  Basal  Medium,  CC-3151,  Clonetics,  San  Diego,  CA)  while  the  third 
dish  r^eived  MEGM  (Mammary  Epithelial  Cell  Growth  Media,  MEGM,  CC-3051,  Clonetics).  These  are  serum 
free  formulations.  MEGM  contains  bovine  pituitary  extract  (BPE),  hydrocortisone,  human  epidermal  growth 
factor  (EGF),  and  insulin,  while  MEBM  does  not.  Thus,  for  the  first  48hrs,  Dish  3  received  all  the  factors 
commonly  required  for  propagation,  while  Dishes  1  and  2  did  not.  MEBM  only  supports  cells  at  a  basal  level  of 
metabolism  and  is  not  intended  to  support  cell  attachment  or  propagation.  After  48hrs,  Dish  #2  was  refed  with 
MEGM  (this  two  step  process  is  referred  to  as  "MEBM-i-EGF"X  Dish  #3  was  also  refed  with  MEGM,  but  Dish  #1 
received  MEGM-hEGF  (BPE,  hydrocortisone,  and  insulin  were  present).  The  Dish  #1  treatment  is  referred  to  as 
"MEBM-EGF".  As  seen  in  fig.  4  of  the  Stampfer  paper  (184B5  cells),  DNA  synthesis  reaches  a  peak 
approximately  18hrs  after  feeding,  with  MEGM  cells  showing  the  greatest  amount  of  DNA  synthesis,  followed  by 
N^BM-i-EGF  cells  which  show  decreased  DNA  synthesis.  Cells  maintained  in  MEBM-EGF  show  little  DNA 
synthesis.  These  results  were  obtained  using  [^H]-thymidine.  In  our  experiments,  we  pulse-labeled  with  BrdU  for 
1  hr,  and  then  dual-stained  the  living  cells  with  H42  and  S16.  One  digital  image  was  acquired  of  cells  with 
360nm  excitation  (H42  signal),  and  a  second  was  acquired  at  490nm  excitation  (S16  signal).  These  images  were 
processed  and  ratioed  on  a  pbcel  by  pixel  basis.  The  resulting  ratio  values  were  then  color  coded  as  shown  in 
Figure  15.  As  seen  in  this  figqre,  our  results  agree  with  those  obtained  by  traditional  l^HI-thymidine 
incorporation  assays.  MEGM  cells  (Dish  3)  showed  the  greatest  degree  of  BrdU  incorporation,  followed  by 
MEBM-i-EGF  cells  (Dish  #2),  with  MEBM-EGF  cells(pish  #1)  showing  little  BrdU  incorporation. 

To  further  verify  these  observations,  the  cells  were  fixed  with  70%  ethanol  and  50mM  glycine,  pH2.0  at  -30 
°C,  and  indirect  immunofluoresence  staining  was  performed.  After  this  staining,  cells  were  restained  with  H42 
and  S16  to  see  if  fixed  cells  labeled  with  the  anti-BrdU  antibcxly  also  had  low  H42  fluorescence,  indicative  of 
BrdU  quenching.  As  seen  in  Figures  7C,D,  there  was  some  evidence  of  this  effect.  Figure  7D  shows  a 
combined  color  image  of  triple-stained  cells.  The  MEGM  cells  exhibited  the  greatest  amount  of  BrdU  staining 
(red  cells  circled  in  white).  If  one  examines  the  same  cells  in  the  blue  channel,  (Figure  7C)  one  can  see  the 
general  trend  that  cells  that  stain  for  BrdU  exhibit  lower  H42  fluorescence  signal. 

Experiment  HM2:  Visualization  of  Intracellular  Calcium  Levels  and  mRNA  Response  in  Living  Cells 

One  goal  of  these  Tasks  was  to  correlate  several  biological  responses  simultaneously  during  VSOM 
experiments.  In  experiment  HM2,  we  dual-stained  184B5  cells  with  FURA-PE3  (green:  a  calcium-sensitive 
fluorescence  probe)  and  LDS-751  (red:  a  DNA  and  RNA  binding  fluorescence  probe  with  a  veiy  different 
emission  wavelengths,  depending  on  whether  it  is  bound  to  DNA  or  RNA).  Cells  were  then  sequentially 
stimulated  with  2pM  thapsigargin  (TG)  and  5p,M  ionomycin  (ION)  (Figures  16,  17).  Both  of  these  compounds 
are  know  to  elevate  intracellular  calcium.  As  seen  in  Figure  16,  the  red  RNA  signal  due  to  LDS-75 1  increases 
greatly  for  all  the  cells  after  intracellular  calcium  has  been  elevated.  The  fluorescence  signal  of  LDS-751  when 
bound  to  DNA  is  emitted  in  the  infrared;  thus,  the  nuclei  of  these  cells  do  not  exhibit  any  red  signal.  The  3D 
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graphs  inset  in  Figure  16  are  zoomed  up  in  Figure  17.  The  LDS-75 1  response  for  all  cells  is  shown  in  Figure 
17A,  while  selected  responses  are  shown  in  Figure  17B.  As  seen  in  the  selected  responses,  some  cells  exhibit  a 
second  response  to  ION  after  the  initial  TG  stimulation,  and  some  do  not.  In  addition,  some  cells  respond  more 
quickly  to  TG  than  do  others.  Our  goal  in  the  next  year  is  to  closely  track  these  cells  and  determine  if  their 
ultimate  biological  fate  (proliferation,  apoptosis,  or  differentiation)  is  closely  correlated  to  perturbations  in 
intracellular  calcium  levels.  In  addition,  we  wish  to  use  VSOM  to  "clamp"  the  intracellular  calcium 
concentrations  (and/or  pH)  to  values  that  will  selectively  induce  proliferation  or  apoptosis  in  different  cell  types. 
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Key  Research  Accomplishments 


•  Demonstrated  the  predicted  VSOM  capabilities 

•  Demonstrated  a  new  class  of  drug  response  and  chemosensitivity  assays 

•  Demonstrated  that  VSOM  is  superior  to  multiwell  plate  MDR  assays 

•  Obtained  two  new  grants  using  VSOM  preliminary  data 

•  Reduced  VSOM  patent  to  practice 

•  Demonstrated  that  living  cells  can  be  segmented  using  transmitted  light  only 

•  Demonstrated  Image  Ratioing  BrdU  Proliferation  Assay  for  Living  Cells 

•  Imaged  mRNA  response  to  high  calcium  in  TG-stimuIated  HMEC  cells 
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Reportable  Outcomes 


Presentations  (included  in  Appendix  C): 

Callahan,  D.E.,  and  Parvin,  B.  Quantitation  of  Single  Cell  Responses  using  Fluorescence  Microscopy. 
Proceedings:  Era  of  Hope  Department  of  Defense  Breast  Cancer  Research  Program  Meeting,  Volume  2,  page 
709.  Atlanta,  GA,  June  8-11,  2000,  Poster  Number:  CC-5. 

Abstracts  (included  in  Appendix  DJ: 

Callahan,  D.E.,  and  Parvin,  B.  Visual  Servoing  for  the  Detection,  Quantitation,  and  Modulation  of  Specific  Cell 
Responses  in  Subpopulations  of  Multidrug  Resistant  (MDRl  Human  Breast  Cancer  Cells,  to  be  presented, 
Biophysical  Society  Annual  Meeting  2001,Boston,  MA,  February  17-21,  2001,  Abstract  Number:  3275. 

Provisional  Patent  Application  (included  in  Appendix  G): 

Callahan,  D.E.,  and  Parvin,  B.  A  Method  for  Knowledge-Based  Discovery  and  Qptimization  of  Differences 
Between  Cell  Types,  LBNL  Provisional  Patent  Application:  submitted:  June  8,  2000, 
reduced  to  practice:  September  22,  2000. 


Grants  Submitted  and  Pending  (Grants  #1,  #2,  #4  (below)  are  included  in  Appendix  G) 

1.  Rapid  Discovery  of  Physiolozical  Characteristics  Which  Distinguish  Malignant  and  Nonmalisnant  Human 
Breast  Epithelial  Cells 

Proposal  ID  number:  BC000508 
Amount:  $505,324 
submitted:  June  6,  2000  to: 

Congressionally  Directed  Medical  Research  Program  2000 
Breast  Cancer  Research  Program 
"Clinical  Bridge  Grant" 

Berkeley  Lab  Proposal  BG  00-297(00) 

Principal  Investigators:  Callahan,  D.E.,  and  Parvin,  B. 

Grants  Submitted  and  Funded; 

2.  Automated  Imasins  System  for  Guidins  Antisense  Comyounds  to  Specific  mRNA  Tarsets  in  Livins  Cells 

Amount:  $1,188,146 
submitted:  May  27,  2000  to: 

DDE  QBER  Program  "Imaging  Gene  Expression  in  Health  and  Disease" 

Berkeley  Lab  Proposal  LABOO-13 

Principal  Investigators:  Parvin,  B.,  Callahan,  D.E.,  Barcellos-HofF,  M.H.,  and  Taylor,  S. 

3.  Quantitative  Spatial  and  Temporal  Resolution  of  Multicellular  Interactions 

Amount:  $240,000 
submitted:  May  16,  2000  to: 

Lawrence  Berkeley  National  Laboratory 

"Laboratory  Directed  Research  and  Development  Grant:  Strategic  Initiative" 

Principal  Investigators:  Barcellos-Hoff,  M.H.,  Callahan,  D.  E.,  Parvin,  B.,  and  Sudar,  D. 
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Grants  Submitted  and  Not  Funded: 


4.  Predictive  Testim  for  the  Individual  Breast  Cancer  Patient 

Amount:  $1,116,897 
submitted:  January  13,  2000  to: 

California  Breast  Cancer  Research  Program 
"New  Scientific  Perspectives  Research  Collaboration" 
Principal  Investigators:  Callahan,  D.E.,  Parvin,  B. 

Berkeley  Lab  Proposal  BG-00-021(00) 


Conclusions 


The  VSOM  demonstrations  we  present  here  clearly  show  the  feasibility  and  advantages  of  VSOM  MDR 
assays.  We  have  demonstrated  a  rapid  VSOM  fluorescence  assay  for  doxorubicin  chemosensitivity  that  can 
identify  a  subpopulation  of  cells  in  a  much  larger  population  of  cells.  In  this  VSOM  fluorescence  assay,  for  this 
model  cell  system,  optimizing  the  retention  of  calcein  is  equivalent  to  optimizing  the  uptake  and  cytotoxicity  of 
the  anticancer  drug  doxorubicin.  It  appears  unlikely  that  a  single  MDR  kit,  single  set  of  inhibitors,  single  set  of 
concentrations  and  exposure  times  will  be  able  to  deal  with  the  wide  range  of  MDR  proteins  that  are 
overexpressed  to  varying  extents  in  multidrug  resistant  tumor  cells.  VSOM  technology  offers  the  advantage  of 
intelligent  adaptation  during  ongoing  quantification  of  dynamic  cell  responses.  This  means  that  an  assay  could  be 
modified  on-line,  intelligently  and  automatically,  on  a  specimen-by-specimen,  or  patient-by-patient  basis.  The 
number  and  type  of  exposures,  the  order  and  duration  of  exposures,  the  compounds  and  the  concentrations  used 
can  be  optimized  during  the  assay  in  order  to  efficiently  and  rapidly  obtain  the  physiological  information 
necessary  to  select  the  best  chemotherapeutic  drugs  for  that  patient.  We  believe  we  have  demonstrated  that  1 
MDR  cell  out  of  1000  DS  cells  can  easily  be  detected  and  stimulated  repeatedly,  for  hours,  in  order  to  determine 
the  number  and  type  of  MDR  proteins  expressed  at  the  cell  surface.  Thus,  we  believe  we  have  demonstrated  the 
technical  feasibility  of  in  vitro,  VSOM  assays  that  could  be  used  to  design  breast  cancer  chemotherapy  for  the 
individual  patient. 
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Appendix  A:  Figure  Legends 

NOTE:  Figures  5E,F  are  the  only  digital  images  presented  in  this  report  that  were  not  acquired  using  a 
lOX,  NA  0.5  Zeiss  Fluar  Objective.  All  images  are  1024x1024,  and  the  CCD  chip  in  our  camera 
has  pixels  of  the  size  6.8x6.8  p.  Thus,  all  images  taken  at  lOX  show  a  field  of  view  that  is 
approximately  700p  high  and  700p  wide. 

Figure  1:  Screen  snapshots  of  the  Java  Command  File  Generator  (JCFG).  (A)  Opening  dialog  box  (B) 
Assignment  of  perfusion  runs  as  a  treatment,  or  exposure  of  a  specific  cell  type  (C)  Assignment  of  a 
group  of  images  to  an  perfusion  interval  (D)  Expanded  tree  view  of  C. 

Figure  2:  A  single  screen  snapshot  of  multiple  Netscape  Browser  windows  that  automatically  open  during  use  of 
the  Java  VSOM  data  browser.  Window  A  presents  a  tree  structure  for  an  image  directory,  and  the 
opposite  frame  in  window  A  displays  a  thumbnail  of  the  ICS  image.  By  clicking  on  Pop-Up  Full-Size 
JPEG,  one  can  bring  up  the  1024x1024  image  in  window  B,  and  by  clicking  on  Show  Image  with 
Segmentation,  one  can  bring  up  the  image  for  a  given  channel  in  window  D  (cell  nuclei  in  channel  0: 
360nm  excitation  of  Hoechst  33342).  If  one  then  clicks  on  an  individual  cell  in  Window  D,  the  time- 
dependent  mean  cellular  fluorescence  is  plotted  (in  Window  C)  for  that  cell  (arrow  in  D)  in  the  current 
channel.  The  calcein  fluorescence  response  seen  in  channel  1  (490nm  excitaion)  is  shown,  and  the 
image  displayed  is  one  acquired  at  an  early  time  point.  The  fluorescence  signal  plotted  in  Window  C 
thus  reflects  calcein  accumulation  in  a  single  cell  over  a  6600s  time  period  (110  min,  or  1.8h).  Data 
from  Experiment  WA3  is  shown. 

Figure  3:  (A)The  functional  architecture  of  the  Deep  View  Bioinformatics  system  is  shown.  Examples  of  (B) 
the  Pump  Log,  (C)  an  XML  recipe  file,  and  (D)  an  ICS  image  header  are  presented. 

Figure  4:  A  schematic  diagram  illustrating  the  use  of  an  etched  culture  dish  for  the  purpose  of  establishing  a 
frame  of  reference  for  cell  location  in  the  dish.  As  discussed  in  Appendix  E,  our  goal  is  to  annotate 
each  cellular  response  with  the  actual  physical  location  of  that  cell  in  the  dish.  This  will  give  us  the 
ability  to  return  to  the  same  field  of  cells  days  later,  in  order  to  determine  the  ultimate  biological  fate  of 
each  cell 

Figure  5:  These  are  examples  of  two  of  the  three  different  image  segmentation  algorithms  that  are  currently  under 
development.  One  is  based  on  a  new  regularized  edge  detector  based  on  supremum.  the  second  on  a 
vector  representation  of  the  underlying  image  followed  by  feature  extraction  in  vector  space. 

(A,B)  Regularized  edge  detector  based  on  supremum:  nuclei  of  living  184B5  cells  stained  with  Sytol6 
are  shown.  The  same  cells  exhibit  mRNA  staining  in  another  channel,  as  seen  in  Figure  16. 

(C,D)  Vector  representation  of  the  underlying  image  followed  by  feature  extraction  in  vector  space 
(fluorescence  image):  Nuclei  of  living  MCF-7  cells  stained  with  Hoechst  33342. 

(E,F)  Vector  representation  of  the  underlying  image  followed  by  feature  extraction  in  vector  space 
(transmitted  light  image):  Phase-contrast  images  of  living  MCF-7  cells  taken  at  20X. 

The  third  image  segmentation  algorithm  is  based  on  a  geometric  and  texture  model  of  a  cell  and  its 
interaction  with  neighboring  cells.  An  example  is  found  on  Page  10  (E)  of  the  Era  of  Hope  Poster 
(Appendix  C). 

Figure  6:  (A)  Schematic  description  of  VSOM  experiment  WA3.  This  experiment  consisted  of  two  VSOM  runs. 
Run  #1  for  MCF-7  WTC  (PS)  cells,  and  Run  #2  for  MCF-7  ADR  (MDR)  cells.  Each  downward 
arrow  indicates  a  different  perfusion  state.  Curved  arrows  indicate  the  extrusion  of  calcein-AM  (CAM) 
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from  the  cell.  Green  indicates  intracellular  accumulation  of  free  calcein  due  to  esterase  cleavage  of  AM 
group  from  CAM.  Blue  indicates  Hoechst  33342  (H42)  -labeled  cell  nuclei.  The  nuclei  of  DS  and 
MDR  cells  are  represented  as  "|||"  and  respectively. 

Compare  this  schematic  to  Figure  9,  where  more  detailed  information  (on  the  syringes,  composition  o^ 
perfusion  solutions,  flow  rates,  and  perfusion  intervals  used  in  the  study)  is  given.  , 

(B)  Preparation  of  multiwell  plates  in  a  sterile  hood  using  a  programmable,  multichannel  pippetor. 

(C)  Multiwell  plate  being  pulled  into  a  temperature-controlled,  fluorescence  multiwell  plate  reader 
(Cytofluar  4000). 

(D)  VSOM  scanning  stage  without  a  stage  plate  mount  inserted.  Silver  areas  represent  modifications 
made  so  that  temperature  controlled  microperfiision  chamber  could  be  mounted. 

(E)  Blow-up  of  a  stage  plate  mount  used  for  chambered  coverslips.  Both  2-well  and  an  8-well 
chambered  coverslips  are  shown  without  lids. 

(F)  A  variety  of  stage  plate  mounts  used  for  different  sizes  of  tissue  culture  flasks,  dishes,  and 
multiwell  plates. 

(G)  Temperature-controlled  microperfusion  chamber  modified  for  mounting  on  the  inverted 
microscope  shown  in  D. 

Figure  7;  Protocols  for  (1)  Cytofluar  4000  Test  (WAD  of  Molecular  Probes  MDR  Assay  p-Test  Kit,  and  (2) 
BrdU  Incorporation  Test  for  Proliferation  in  Living  Cells  (HMl) 

(A) :  Layout  of  plates  for  the  MDR  fluorescence  assay  (experiment  WAl)  that  employed  living  cells  in 
24-well  plates.  Each  treatment  group  appears  in  quadruplicate.  A  concentration  ladder  of  cells,  and 
blank  wells  are  also  shown.  Results  appear  in  Table  1,  page  12. 

(B) :  Flow  chart  representation  of  the  protocol  used  for  the  BrdU  ratio  imaging  test  (experiment  HMl). 

(C) :  The  blue  channel  of  the  color  image  shown  in  D  (HMl).  Fixed  eells  (nuclei)  stained  with  H42  are 
shown.Red  nuclei  that  are  circle  in  white  in  D  are  circled  in  blue  in  this  figure. 

(D) :  Fixed  and  triple-stained  184B5  cells  from  HMl.  Cells  were  pulse-labeled  with  BrdU  for  one  hour. 
Three  digital  images  were  obtained  on  the  red,  green,  and  blue  channels  (570,  490,  and  360nm 
fluorescence  excitation,  respectively).  These  three  images  were  combined  into  the  full  color  image 
shown  here.  Red  fluorescence  indicates  indirect  immunofluorescence  detection  of  incorporated  BrdU 
using  an  anti-BrdU  antibody.  Green  fluorescence  represents  the  Sytol6  nuclear  stain  (S16,  which  also 
stains  RNA  to  some  extent).  Blue  fluorescence  represents  the  nuclear  stain  Hoechst  33342  (H42). 
From  these  digital  images,  there  are  two  ways  to  quantify  BrdU  incorporation:  (i)  anti-BrdU  antibody 
detection,  represented  by  the  intensity  of  red  fluorescenee  (ii)  calculation  of  the  ratio  of  blue  nuclear 
fluorescence  intensity  (H42)  divided  by  green  nuclear  fluorescence  intensity  (S16)  on  a  pixel  by  pbcel 
basis  (image  ratioing).  H42  is  quenched  by  BrdU  incorporation,  while  S16  is  not.  Thus,  the  nuclei  that 
appear  red  here  (circled  in  white),  should  exhibit  lower  H42  fluorescence  in  the  blue  channel  (C),  where 
they  appear  circled  in  blue. 

Figure  8:  Digital  Imaging  Test  of  Molecular  Probes  MDR  Assay  P-Test  Kit  (Experiment  WA2).  The  protocol 
shown  in  Figure  6A  (also  used  in  experiment  WAl,  Cytofluar  4000  Test  of  MDR  Assay  P-Test  kit) 
was  repeated  here  using  cells  plated  in  four  35mm  circular  cell  culture  dishes.  In  this  color-coded 
representation  of  gray-scale  images,  low  intensity  pkels  have  been  assigned  "colder"  colors  (black, 
purple,  blue)  while  "warmer"  colors  (yellow,  red,  white)  have  been  assigned  to  the  higher  intensity 
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pixels.  As  shown  in  the  bar,  the  warmer  colors  indicate  higher  levels  of  intracellular  calcein.  (A)  MDR 
(MCF-7  ADR)  incubated  with  CAM  (only).  These  cells  accumulate  relatively  little  calcein.  (B)  MDR 
cells  incubated  with  CAM+MK.  (C)  MDR  cells  treated  with  CAM+V  (D)  Image  of  DS  (MCF-7  WTC) 
cells  stained  with  CAM  (only).  These  digital  images  show  good  agreement  with  the  expected  behavior 
of  the  P-Test  kit  assay  (Figure  6A). 

Figure  9:  This  diagram  gives  detailed  information  on  the  syringes,  composition  of  solutions  in  the  syringes,  flow 
rates,  and  perfusion  intervals  used  in  VSOM  experiment  WAS.  This  experiment  consisted  of  two 
VSOM  runs.  Run  #1  for  MCF-7  WTC  (DS)  cells,  and  Run  #2  for  MCF-7  ADR  (MDR)  cells.  These 
two  runs  were  performed,  one  after  the  other,  on  the  same  day.  Both  runs  used  the  pump  schedule.  At 
the  end  of  each  run,  the  plot  which  was  displayed  on  the  computer  screen  was  saved.  The  plots  for 
Runs  #1  and  2  are  shown  in  black  windows  at  the  botton  of  the  figure.  The  red  line  in  each  case 
represents  the  average  of  all  the  cell  responses  in  the  entire  field  of  view.  As  seen  in  these  two  plots 
recorded  for  MCF-7  WTC  (DS,  run  #1)  and  MCF-7  ADR  (MDR,  run  #2),  DS  cells  (upper  plot) 
accumulated  more  calcein  (240±82,  N=336)  than  did  the  MDR  cells  (103+80,N=161).  This  is  in 
agreement  with  the  relative  differences  in  calcein  accumulation  observed  between  the  same  DS  and 
MDR  cells  in  the  multiwell  plate  experiments  which  represent  an  average  over  many  cells. 

Figure  10:  The  individual  cell  responses  of  each  DS  cell  in  Run  #1  (Experiment  WAS)  are  shown  in  green.  The 
mean  response+standard  deviation  of  all  the  cells  is  shown  overlaid  in  black.  During  VSOM 
experiments  the  mean  cellular  fluorescence  of  each  cell  is  monitored,  and  deviations  from  the 
population  mean  can  easily  be  detected.  For  example,  the  arrow  indicates  several  cells  that  began  to 
lose  viability  and  plasma  membrane  integrity.  Such  cells  are  unable  to  retain  calcein  and  the 
fluorescence  signal  drops.  It  should  be  noted  that  (1)  the  largest  number  of  cells  observed  to  date  in  a 
single  field  is  1000  (2)  the  current  VSOM  system  could  easily  terminate  or  modify  the 
exposure/perfusion  as  soon  as  a  single  cell  death  is  noted  (3)  the  current  VSOM  system  could  easily 
detect  a  single  MDR  cell  (Figure  11,  red  curves)  out  of  1000  DS  cells.  This  is  because  there  is  no 
overlap  between  such  cell  responses  between  1000  and  5000s. 

Figure  11:  Selected  individual  cell  responses  from  four  cells  in  Run  #2  (Experiment  WA3).  This  is  the  type  of 
detailed  physiological  information  that  flow  cytometric  assays  and  multiwell  plate  assays  catmot 
provide,  and  it  demonstrates  that  our  VSOM  technology  has  the  capabilities  we  initially  predicted. 
Detailed  rate  information  on  calcein  accumulation  and  retention  is  available  throughout  the  VSOM 
experiment.  For  the  blue  response  curves,  one  can  calculate  a  rate  of  calcein  accumulation  during  the 
three  CAM  perfusions,  and  one  can  calculate  a  rate  of  calcein  retention  during  BUFF  perfusions. 
Calcein  accumulation  is  occurring  in  the  1500,  3500,  and  5500s  time  periods,  and  plateaus 
corresponding  to  excellent  calcein  retention  are  observed  in  the  2500,  4500,  6500  time  intervals.  For 
the  red  curves,  no  calcein  accumulation  occurs  until  5000s,  when  CAM+V  is  perfused. 

Figure  12:  Calcein  accumulation  in  65  MCF-7  ADR  (MDR)  cells  during  Calcein-AM  (CAM)  perfusion  (VSOM 
Experiment  WA4).  CAM  (at  the  four-fold  higher  concentration  of  l.OpM)  was  perfused.  The  first 
perfusion  interval  this  experiment  is  shown,  and  the  same  data  is  presented  using  a  variety  of  3D 
perspectives.  As  seen  earlier  (Figure  11,  blue),  a  significant  subpopulation  of  calcein-accumulating 
(DS)  cells  is  present,  and  these  cells  demonstrate  varying  degrees  of  calcein  accumulation  during  a 
2100s  exposure.  Portions  of  this  experiment  are  presented  in  the  Biophysical  Society  Abstract 
(Appendix  D).  Cells  that  accumulate  calcein  are  not  multi-drug  resistant.  Four  views  of  the  same  data 
are  shown.  (A)  Selected  responses  of  14  cells  that  accumulated  calcein.  3D  response  curves  are  color 
coded  according  to  intensity  (B)  All  65  cell  responses  shown  (C)  All  65  cell  responses  viewed  end-on 
(D)  2D  projection  of  data  shown  in  3D  in  C. 

Figure  13:  MCF-7  Cells  vs.  HMECs:  pH  is  a  function  of  CO2,  Buffering  System(s)  and  Metabolism.  MCF-7 
cells  (WTC  and  ADR,  blue  circles)  grown  in  DMEM  (containing  a  bicarbonate  buffer)  and  HMECs 
(red  triangles)  grovm  in  MEGM  (containing  both  a  bicarbonate  buffer  and  a  HEPES  buffer)  show 


Annual  Report:  Callahan,  Parvin  Page  23 


different  pH  behavior  as  a  function  of  incubator  CO2  levels.  As  indicated  by  the  vertical  arrows,  the 
pH  range  tolerated  by  each  cell  type  varies.  The  pH  zone  common  to  both  cell  types  is  shown  in 
green.  There  is  not  a  common  CO2  level  that  allows  these  different  cell  types  (in  separate  flasks,  in  the 
media  that  each  prefers)  to  simultaneously  propagate  in  a  single  incubator.  The  more  normal  HMECs 
tolerate  a  much  narrower  range  of  pHs  than  do  the  MCF-7  cancer  cells.  None  of  the  cell  lines  prefer  a 
lower  pH  environment  (<7.1). 

Figure  14:  Transmitted  light  images  of  MCF-7  WTC  cells  propagated  in  media  of  varying  pH.  Images  were 
acquired  by  placing  a  T-25  tissue  culture  flask  on  the  microscope  using  the  appropriate  stage  mounting 
plate  (Figure  6F). 

(A,B)  MCF-7  WTC  cells  incubated  at  CO2  levels  that  produce  a  pH  >7.4  (on  average)  in  DMEM 
successfully  attach,  spread  out,  and  proliferate  on  the  plastic  tissue  culture  ware  surface. 

(C  J))At  pHs  <  7.2,  MCF-7  WTC  cells  round  up,  and  do  not  do  attach  well.  A  similar  effect  is  seen  if 
MCF-7  ADR  are  plated  under  conditions  where  the  pH  >  8.0. 

Figure  15:  Color  coded  ratioed  images  of  184B5  cells  (Experiment  HMl).  Living  cells  were  pulse-labeled  with 
BrdU  for  1  hr,  and  then  dual-stained  with  H42  and  S16.  One  digital  image  was  acquired  of  cells  using 
360nm  excitation  (H42  signal),  and  a  second  was  acquired  using  490nm  excitation  (S16  signal). 
These  images  were  processed  and  ratioed  on  a  pixel  by  pixel  basis.  The  resulting  ratio  values  were 
then  color  coded  as  indicated  by  the  bar.  MEGM  (complete)  cells  showed  the  greatest  amount  of  DNA 
synthesis,  followed  by  MEGM+EGF  cells  which  show  decreased  DNA  synthesis.  Cells  maintained 
in  MEGM-EGF  show  little  DNA  synthesis.  Our  results  agree  with  those  obtained  by  traditional  l^FTI- 
thvmidine  incorporation  assays. 

Figure  16:  Experiment  HM2:  HMECs  (184B5  cells)  were  dual  stained  with  FURA-PE3  (green:  a  calcium- 
sensitive  fluorescence  probe)  and  LDS-751.  Cells  were  sequentially  stimulated  with  2pM  thapsigargin 
(TG)  and  5pM  ionomycin  (ION).  Both  of  these  compounds  are  know  to  elevate  intracellular 
calcium.  The  red  mRNA  signal  due  to  LDS-751  increases  greatly  for  all  the  cells  after  intracellular 
calcium  had  been  elevated.  The  fluorescence  signal  of  LDS-751  when  bound  to  DNA  is  emitted  in  the 
infrared;  thus,  the  nuclei  of  these  cells  do  not  exhibit  any  red  signal.  The  3D  graphs  inset  in  Figure  16 
are  zoomed  up  in  Figure  17. 

Figure  17:  Experiment  HM2:  (A)  The  LDS-751  response  for  all  cells  is  shown  in  Figure  16,  (B)  selected 
responses  taken  from  panel  A.  As  seen  in  the  selected  responses,  some  cells  exhibit  a  second  response 
to  ION  after  the  initial  TG  stimulation,  and  some  do  not.  In  addition,  some  cells  respond  more 
quickly  to  TG  than  do  others. 
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(A)  Functional  Architecture 


< Pump  state 

current  time  secs="96570l306" 
current  time  usecs="6466(l” 
pumpnumber="0" 
syringe  0  rate="2. 100000" 
syringe  0  rate  units="ml/mn" 
syringe  1  rate="2. 700000" 
syringe  I  rate_units=" ml/m n" 
syringe  O  tlianietcr="26.000000" 
syringe_l_diameter="26.000000" 
running  or  stopped="Stopped" 
mode="  Proportional" 
directi«n="  Parallel" 

/> 

<Pump_state 

current  timc_sccs="96570 1 306" 

curreiit_time_usccs="374 1 90" 

pumpnumbcr="l " 

syringe_0_rate=”2.300000" 

syringe_0_rate_units="ml/mn" 

syringe_  I  _rate="0.000000" 

syringc_l_rate_units="ml/mn" 

syringc_0_diamctcr="27.000fl00" 

syringc_l_diaincter="25.000fl00" 

running_or_stopped="Stopped" 


<S<MrkArqai<iti(in 

SINCLF:_or_MLLTIPl.E_Mack_jti«|U»itti.n="Ml'LTIPLE" 

TOTAL_TIME_«r  M  VI  STACKS. Jo_ali)|»  niMltiplf. Mack. act|uiyi»ion="TOTAl,_TlME" 
HiUl_linif_yit«nnl*_lhvn_'t<ip="72«ir 


<(!'hanncl 

filt«rWhi-4'IPo!>iti<iii="<l'' 
hcil(icrInfl>_na^i■k'n^th="3f)<>^m'' 
hcadcrinkt  fik-iiiiitH-ChHnnvK'har°''4l" 
dci»cri|)tk»n="H42" 

ACTIVE=".tty" 

<f\|i«i«urcTlnK'_in«>  2l*ll</e\po'«iccT»mc_nn> 
</C’hanncl> 

<C'fianncl 

filt4TW  hctlP«i»jtii)n=’'  > " 
hcadcrintn  «avckn(jlh="-l'>tlnin" 
luadirlnfnfUcnamcChanncIChar^"  I " 
dt-icripii(in="iHiM'' 

ACHVE='Sf,-> 

<c\poiuivTimc_in«>  l(HHI</c\pi>siirTTink.'_im> 
<^Ch>inncl> 

<Channcl 

filUrWhtt'IP«illiiin="2" 

header  lftr«_«»sck-njjlh="57Hnm’' 

lM-adcrlnf»_rik-numc('hunnc1Char="2" 

ik'scriplMin=’'nuH‘’ 

ACTIVE=-'>cs"> 

<4;\p»MirvTimc_n«'*  1llim</e\pinurcT1inc_mti> 
</Chaflni'l> 

^Channel 

(UlcrW  hcvlPt)\ilii>n="J" 
h(-aik-rlnri>_HattrknKth="3H<lnm" 
header  tnf<i_rik'nami'C  lMnnclC'har="3" 
dc<tiripii»n°"nuir' 

ArTl\F.®”>«"»"> 

<c\p«)surcTin»c  in«>  5MIK/i-\pii»upcTimi-  m<t> 
</Chai»f*tl> 

<Channel 

riJHr\MKtlP..yilN.n='4-’ 
lH'a<k'rtnfi>_wayi-k'ngth=''34llnni'' 
kL-atkrlnfo  fik  namcChanm  lC  har:’'4" 
(k'ycriptH>n="nuir 
ACT>VE-*>e»'> 

<*Apt)»ureTimi_ms>  50tl<,V\p(>suriTlmt_mN> 
</ChamKt> 

<rhanrH‘l 

nilerWheeJPnykiiin="f" 
headtrlnr»i_waseknuth*"OPEN” 
header  lnf<>_fik'nami’C'hani>clChar""5" 
de-Mripik>n»”nuir’ 

A<TI\E-”bo“> 

<e\piicurrTlnK_ni‘>  ll>«</f\piiHureTlm4_m%> 
<.'Ch«nnfl> 

<Chai»ni’l 

BlurWhtflP»Nilh>n=”f)’' 
hcaihTlnft>_«aulcr>(>lh=‘’TRANS" 
header  lnf»_nk'nanH’Channt'IC'hai«"\'' 
<k'seripiiun*”l>riKhlfiehr' 

ACTIVE=">e4-> 


(B)Small  Selection  from  "Pump  Log" 


(C)  XML  "Recipe"  File 


icsversion 

1.0 

filename  dc051 

.lOOdc.OLI 

layout  parameters  4 

layout  order 

bits 

X 

y 

2 

layout  si/es 

16 

!024 

1 024 

5 

layout  coordinates 

video 

layout  significant  bits 

16 

representation 

format 

integer 

representation 

sign 

signed 

representation 

compression 

com|>iess 

representation 

b\le  order  2 

I 

representation 

SCIL  TYPL 

g3d 

parameter 

ongin 

0 

1 60 

6 

0 

parameter 

scale 

I 

0.6800000 1 

0.68000(KH  0 

parameter 

labels 

intensity 

x-pos 

y-pos 

probe 

parameter 

units 

relative 

microns 

microns 

undefined 

parametei' 

exptype 

beads 

iparametei’ 

exptarget 

other 

parameter 

probe 

dapi  360 

fitc  490 

tr  570 

Iura2  380  fura2  340 

parameter 

target 

other 

otlier 

other 

odier  other 

parameiei' 

exposure 

rime 

LOO 

LOO 

LOO  LOO  LOO 

history  date 

Sat  May  I 

I.-?  23:50:22 

2000 

histor>  computer  Sun  Sparc  lltra  170 

historv  laboratorv  l.BNI.  Lite  Sciences  Division 

historv'  opeiator  Darnel  F  Callahan 

history  microscope  Zeiss_Axioverll35TV 

historv  objective  Zeiss  Fluar  10x(l.50NA 

historv’  camera  dev  sdvO 

history  softwrtre  SCIL-lmage  L3 

history  comments  New  Xc  latnp,  base  incHintcd  camera 


FIGURES 


(D)  ICS  Image  Header  File 
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KEEP  TRACK  OF  CELL  LOCATIONS 
(LOCATION  OF  MICROSCOPIC  FIELD 
OF  VIEW)  DURING  EXPERIMENTS 


O  ®  •  DIFFERENT  TYPES  OF  LIVING 

9  ^  CELLS. 


(x.y)  ONE  FIELD  OF  VIEW  IN  A  PETRI 

DISH.  KEEP  TRACK  OF  RELATIVE 
1024x1024  POSITION  OF  CELLS  DURING  TIME 

LAPSE  EXPERIMENT.  BE  ABLE  TO  RETURN 
TO  THIS  FIELD  OF  VIEW  DAYS  LATER. 


FIGURE  4 
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FIGURE  6 


V  MK 


CONTROL 


Layout  of  the  four  plates  used  in 
experiment  WAl.  (Control,  CAM 
(only),  4  wells;  V,  CAM+ Verapamil, 
4  wells;MK,  CAM+MK-571,  4  wells; 
A1  and,  D6  are  array  designations 
for  2  of  the  4  blank  wells.)  Black 
dots  in  each  well  represent  relative 
cell  density. 
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BrdU  Incorporation  Assay 


FIGURE  7 
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FIGURE  8 
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MCF-7  ADR:  Modulation  of  MDR.  Four  Individual  Cell  Responses  Shown. 
RED  :  Cells  that  accumulated  Calcein  only  when  Verapamil  was  present. 
BLUE:  Cells  that  demonstrated  additional  responses. 
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FIGURE  :  Calcein  accumulation  in  65  MCF-7adr  cells  during 
Calcein-AM  perfusion.  Cells  that  accumulate  calcein  are  not  multi¬ 
drug  resistant.  Four  views  of  the  same  data  are  shown.  (A)  Selected 
responses  of  14  cells  that  accumulated  calcein.  3D  response  curves 
are  color  coded  according  to  intensity  (B)  All  65  cell  responses  shown 
(C)  All  65  cell  responses  viewed  end-on  (D)  2D  projection  of  data 
shown  in  3D  in  C. 
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pH  is  function  of  C02,  Buffering, 
System(s)  and  Ceii  Metaboiism 
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A.  MCF-7  WTC  c.  MCF-7  WTC 
in  DMEM,  pH  >  7.2  in  MEGM,  pH  <  7.2 


B.  MCF-7  WTC  D.  MCF-7  WTC 
in  DMEM,  pH  >  7.2  in  MEGM,  pH  <  7.2 


FIGURE  14 
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LIVING  CELLS  STAINED  WITH  NUCLEAR  STAINS  Hoechst  33342  and  Syto  16 


First  48  h: 
Next  20  h; 


BrDU 

Incorp. 


MEBM  -  EGF 
MEBM  -  EGF 


MEBM  -  EGF 
MEBM  +  EGF 


MEGM,  complete 
MEGM,  complete 


Low 

BrdU 

Drp. 


FIGURE  15 
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Single  cell  responses  of  225  184B5  cells  These  cells  were  shown  in  the  two  previous 
images  at  t=473  sec  and  t=  2645  sec.  A  heterogeneous  set  of  responses  can  be  seen. 
Four  of  these  responses  are  colored  black  and  and  shown  in  greater  detail  below. 


Selected  cell  responses  to  (a)  RED:  2  pM  thapsigargin(TG)  (473s,  1  inL/min,  45  mL) 
and  (b)  (.  iRl  I  N:  5  pM  ionomycin  (3941s,  1  mL/min,  15  inL).  Some  cells  exhibit 
a  second  response  to  ionomycin  174,  255),  and  some  do  not  (56,  149).  In  addition. 
Some  cells  (225)  respond  more  quickly  to  TG  than  others  (56,149,  174). 


FIGURE  17 
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Appendix  C:  Era  of  Hope  Poster 


Callahan,  D.E.,  and  Parvin,  B.  Quantitation  of  Single  Cell  Responses  using  Fluorescence  Microscopy. 
Proceedings:  Era  of  Hope  Department  of  Defense  Breast  Cancer  Research  Program  Meeting,  Volume 
2,  page  709.  Atlanta,  GA,  June  8-11, 2000,  Poster  Number:  CC-5. 
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ABSTRACT 

The  purpose  of  this  two-year  IDEA  proposal  is  to  gather  preliminary  data  that 
demonstrate  the  feasibility  of  a  new  class  of  in  vitro  drug  response  and 
chemosensitivity  assays.  Our  objectives  are  to  develop  an  instrument  (a  visual 
servoing  optical  microscope,  or  VSOM)  in  a  stepwise  fashion  and  gather 
preliminary  data  to  aid  future  development  of  cell  culture  techniques  and 
fluorescence  assays.  Our  goal  is  to  demonstrate  the  feasibility  of  using  VSOM 
technology  to  analyze  the  heterogeneity  and  physiological  characteristics  of  tumor 
cells  from  individual  breast  cancer  patient  biopsies.  We  believe  that  the  VSOM 
will  be  extremely  useful  in  developing  new  VSOM  chemosensitivity  fluorescence 
assays  for  a  variety  of  breast  cancer  drugs.  VSOM  fluorescence  assays  are  based  on 
the  dynamic  physiological  responses  of  single  cells.  In  these  assays,  computer 
controlled  syringes  inject  various  substances  into  a  temperature  controlled 
microperfusion  chamber.  We  are  currently  observing  the  single  cell  responses  of  up 
to  500  cells  in  five  separate  channels.  The  following  cell  types  have  been  examined; 
MCF-7  (human  breast  cancer  cell  line,  drug  sensitive,  or  DS),  MCF-7adr  (multi¬ 
drug  resistant  ,  or  MDR,  version  of  MCF-7),  184SK  (normal  human  mammary 
epithelial  cells,  or  HMECs),  and  184B5  (immortally  transformed  HMECs).  At  the 
present  time,  we  have  quantified  the  uptake  and  retention  of  the  fluorescent 
compounds  FURA-2AM,  calcein-AM,  Lysotracker-Red,  and  Hoechst  33342  in  all 
the  cell  types.  We  have  also  documented  the  effects  of  ionomycin,  thapsigargin,  and 
4-Br-A23187  in  184SK  and  184B5  cells  labeled  with  FURA-2AM  and  LDS-751. 
We  have  the  infrastructure  in  place  to  handle  very  large  amounts  of  digital  video 
data  and  perform  on-line  analysis  and  intelligent  instrument  control.  Once  the 
VSOM  is  fully  automated  it  will  be  extremely  useful  for  dissecting  complex 
biochemical  pathways,  isolating  and  optimizing  important  environmental  variables, 
and  designing  fluorescence  assays  for  specific  cell  types  or  for  breast  cancer  drug 
chemosensitivity. 

The  U.S.  Army  Medical  Research  and  Materiel  Command  under  DAMD17-98-1- 
8177  supported  this  work. 
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Tumor  cells  from  breast  cancer  patient 

(Surgical  Discard,  FNA) 

i 

k 

Isolation  and  Culture  of  Primary  Breast  Tumor  Cells 

• 

(Dairkee,  et  3[.,Cancer  Res.,  57,1590  (1997),  Li,  et  a\.,Cancer  Res.,  58,  5271  (1998)) 

i 

k 

i 

O 

Multiwell  Plate  Analysis 
under  VSOM-Optimized 
Environmental  Conditions 

888880 
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(A)  Microperfusion  Chamber  With  Probes  and 
Vacuum  Aspirator 

(B)  Computer-controlled  xy  scanning  stage  with 

plate  insert. 
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VSOM:  Zeiss  Inverted  Fluorescence  Microscope  with  FourComputer- 
Controlled  Syringe  Pumps  leading  to  a  temperature-controlled  micro¬ 
perfusion  chamber  mounted  on  a  computer-controlled  xy  scanning  stage. 
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Visual  Servoing  Optical  Microscopy  (VSOM) 

for 

A  New  Class  of  Predictive  In  Vitro  Drug  Response 
and  Chemosensitivity  Assays 

•  Hundreds  of  individual,  living  cells  closely  monitored 

•  Quantification  of  dynamic  physiological  responses  in  multiple 
channels 

•  Real-time  digital  image  segmentation  and  analysis 

•  Intelligent,  repetitive  computer-applied  cell  stress  and  cell 
stimulation 

•  Ability  to  return  the  same  field  of  cells  for  long  term  studies 
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LIVING  CELLS:  DNA  in  nucleus  labeled  with  blue  fluorescent  dye 
(A)  A  two-channel  digital  image:  (i)  transmitted  light  +  (ii)  blue 
fluorescence  (B)  One  channel  image:  blue  fluorescence  only 
(human  mammary  epithelial  cells  at  lOX,  Hoechst  33342). 
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LIVING  CELLS:  Cytoplasm  labeled  with  green  cell  viability  dye. 
(A)  A  two-channel  digital  image:  (i)  transmitted  light  +  (ii)  green 
fluorescence  (B)  One  channel  image:  green  fluorescence  only 
(human  mammary  epithelial  cells  at  lOX,  Calcein-AM). 
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LIVING  CELLS:  Lysosomes  labeled  with  acidoptropic  red  dye. 
(A)  A  two-channel  digital  image:  (i)  transmitted  light  +  (ii)  red 
fluorescence  (B)  One  channel  image:  red  fluorescence  only 
(human  mammary  epithelial  cells  at  lOX,  Lysotracker-Red). 
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CELL  DEATH  DURING  UV  IRRADIATION:  Cytoplasm  labeled 
with  green  and  red  cell  viability  dyes.  Nucleus  stained  with  blue  dye. 
Four  channel  images  shown  (red,  green,  blue,  and  transmitted  light. 
(A)  All  cells  viable  (B)  Membrane  blebbing  and  loss  of  red  and  green 
cell  viability  stains,  nuclei  still  blue.  Dynamic  process  was  monitored, 
(human  cancer  cell  line,  MCF-7  at  lOX,  Hoechst  33343,  Lysotracker- 
Red,  Calcein-AM,  brightfield  transmitted  light). 
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ADVANCED  CELL  DETECTION  AND  TRACKING:  We  continue  to 
improve  our  quantitative  image  processing  and  image  segmentation 
algorithms.  (A-D)  Test  case  for  vector  representation  of  digital  images 
and  feature  extraction  in  vector  space  (vortex  detection). 
(E)  MCF-7  cells  stained  with  TMRE,  a  mitochondrial  and  cytoplasm 
stain  useful  for  monitoring  the  membrane  potential  of  the  mitochondrial 
and  plasm  membrane  (63X,  oil  immersion  objective).  Our  goal  is  non- 
invasive  segmentation  of  each  cell  and  its  important  subcompartments  and  organelles. 
This  is  demonstrated  for  the  nuclear  and  cytoplasmic  compartment  in  E. 
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DETECTION  AND  QUANTITATION  OF  FLUORESCENCE  INTENSITY  IN  THE 
NUCLEAR  SUBCOMPARTMENT  OF  INDIVIDUAL  LIVING  CELLS. 
Rapid  and  accurate  image  segmentation  is  required  for  VSOM  experiments.  In  order 
to  monitor  physiological  responses  in  individual  cells,  we  often  quantify  changes  in  the 
mean  fluorescence  intensity  of  individual  cells  or  cell  compartments. 
(A)  This  is  one  frame,  and  one  channel  (blue)  of  a  time-lapse  experiment.  These  are 
the  nuclei  of  the  blue  channel  for  the  MCF-7  cells  shown  in  Figure  9B.  (B)  Nuclei  of 
detected  using  the  vortex  method  shown  in  Figure  10  A-D. 
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Fluorescence  Intensity  (Arbitrary  Units,  0-4096) 
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Reduced  uptake  and  retention 
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Physiological  Responses  of  hundreds  of  living  ^ 
MCF-7  cells  quantified  in  three  separate  channels.  ” 
(A)  Mean  fluorescence  intensity  of  individual  cells  \ 
+/-  standard  deviation  for  blue,  green,  and  red 
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A 


B 


D 


E 


Stimulation  of  mRNA 
production  in  living 
cells:  Human  mammary 
epithelial  cells  were 
stimulated  with  TG 
(thapsigargin)  at  t=0. 

(A-C)  Time  series  in 
red  ehannel  (LDS-751). 

(D)  Staining  of  nuclei 
in  green  ehannel  using  Sytol6.  (E)  Detection  of  cells  using  vortex  method. 
Time  elapsed  between  A-C  is  approximately  2  hrs. 


Page  13  of  15 


Annual  Report:  Callahan,  Parvin  Page  56 


DOD  BCRP  Poster,  Atlanta,  June  2000 

Conclusions 

1)  VSOM  assays  are  feasible  as  originally  envisioned. 

2)  We  are  now  ready  to  undertake  several  new  steps  in  the 
continuing  development  of  VSOM  technology,  assays, 
and  optimized  microenvironments. 

a)  Revisit  the  same  field  of  cells  after  cells  have  been 
returned  to  a  controlled  environment  for  long-term 
incubation.  This  is  necessary  for  verifying  the  predictive 
power  of  early  cell  responses. 

b) Move  from  established  cell  line  models  to  new  studies  on 
primary  breast  tumor  cells. 

c) Peform  automated  “clamping”  perfusion  studies  for 
precise  control  of  intracellular  ionic  environment  (e.g., 
intracellular  calcium  levels). 

d)  Design  microenvironments  that  increase  proliferation  rate 
of  primary  breast  tumor  cells  in  culture. 

e)  Verify  VSOM  results  using  multiwell  plate  systems. 

f)  Begin  to  populate  a  modem  object-oriented  database  with 
single  cell  responses  to  stress  and  various  stumuli 

g)  Perform  Multi-Field  (MF)  VSOM  in  order  to  monitor 
several  fields  of  cells  at  the  same  time. 
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Appendix  D:  Biophysical  Society  Abstract 


Callahan,  D.E.,  and  Parvin,  B.  Visual  Servoing  for  the  Detection.  Quantitation,  and  Modulation  of 
Specific  Cell  Responses  in  Subpopulations  of  Multidrug  Resistant  (MDR)  Human  Breast  Cancer  Cells, 
to  be  presented.  Biophysical  Society  Annual  Meeting  2001,Boston,  MA,  February  17-21,  2001,  Abstract 
Number:  3275. 
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VISUAL  SERVOING  FOR  THE  DETECTION,  QUANTITATION,  AND 
MODULATION  OF  SPECIFIC  CELL  RESPONSES  IN  SUBPOPULATIONS  OF 
MULTIDRUG  RESISTANT  (MDR)  HUMAN  BREAST  CANCER  CELLS. 

Daniel  E.  Callahan,  Bahrain  Parvin:  Lawrence  Berkeley  National  Laboratoiy,  Cell  and 
Molecular  Biology  Department,  1  Cyclotron  Road,  Berkeley,  CA  94720 
MDR  can  be  dependent  on  transmembrane  proteins  (e.g.,  PgP  and  MRP)  that  extrude 
foreign  compounds.  Expression  of  these  proteins  is  indicated  by  reduced  accumulation 
of  calcein.  An  automated  digital  imaging  fluorescence  microscope  was  used  to  perfuse 
calcein-AM  (CAM,  0.25pM)  containing  modulating  agents  MK-571  (lOpM)  or 
verapamil  (50pM)  into  a  microperfusion  cell  chamber.  MK-571  is  a  specific  inhibitor  of 
MRP,  while  verapamil  inhibits  both  PgP  and  MRP.  MCF-7ADR  cells  labeled  with 
Hoechst  33342  were  observed  at  35°C.  Mean  fluorescence  intensity  per  cell  (MI, 
calcein)  was  calculated  in  real-time  and  software  operation  of  syringe  pumps  was  based 
on  these  calculations;  hence,  the  term  visual  servoing.  The  protocol  found  in  an  MDR 
Beta-Test  Kit  (Molecular  Probes,  Eugene,  OR),  was  modified.  Images  were  acquired 
during  perfusion  intervals  (1)  0-2 100s,  CAM,  (2)  8600-1 1000s,  CAM+MK-571,  and  (3) 
14000-1 6000s,  CAM+verapamil.  All  cells  responded  by  3.  Subpopulationsexpressing 
MRP  and/or  PgP  can  be  inferred  based  on  responses  where  AM1>200  (FIGl):  (a) 
Responses  in  1  (no  PgP  or  MK-571)  (b)  None  until  2  (MRP)  (c)  None  in  1  or  2  (Pgp 
only)  Funded  by  Department  of  Defense,  DAMDl  7-98- 1-8 177  &  U.S.  Department  of 
Energy  DE-AC03-76SF00098. 


CELL  NUMBER  LABEL 

FIGURE  1:  CAM+MK-571  nerfiision.  interval  2 18600  -  1  lOOOsI. 
The  2D  projection  of  MI  versus  time  for  69  cells  is  shown. 

Time  axis  is  perpendicular  to  the  plane  of  the  page.  The 
increase  in  MI  due  to  calcein  accumulation  was  greater  than 
200  in  30  cells,  due  to  inhibition  of  MRP  by  MK-S71.  Three 
digital  images  (channels)  were  acquired  every  60s.  The  MI 
of  cell  #10,  for  example,  increased  from  30  to  760  (arrow) 
during  this  2400s  perfusion  interval 
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Appendix  E:  Imaging  Protocol  for  Returning  to  the  same  set  of  cells  Repeatedly 

I.  Goal  of  these  experiments: 

The  goal  of  these  experiments  is  to  (i)  make  observations  on  one  or  more  sets  of  living  cells  in  a  cell  culture  dish 
on  a  moveable  microscope  stage,  (ii)  remove  the  dish  (moving  the  stage  during  the  interim)  and  return  it  to 
the  incubator,  (iii)  replace  the  dish,  (iv)  automatically  return  to  the  same  set  of  cells,  (v)  make  more 
observations,  (vi)  remove  dish,  and  (vii)  repeat  this  process  as  many  times  as  desired.  The  responses  of  the 
set  of  cells  (500-1000  cells  at  lOX  magnification)  are  observed  as  various  compounds  are  perfused  into  the 
dish.  A  stack  of  images  (consisting  of  a  specified  number  of  channels)  is  acquired  one  or  more  times  at  a 
specified  intervals  for  a  specified  length  of  time.  An  editable  (text)  recipe  file  specifies  experimental 
parameters.  For  “snapshots”  only  one  image  stack  is  acquired.  For  “time-lapse”  experiments,  a  complete 
stack  of  images  is  acquired  at  regular  intervals.  Several  time-lapse  experiments  may  be  performed  (in 
sequence)  on  the  same  set  of  cells  before  removing  the  dish  (user  should  not  move  stage  or  dish  between  these 
sequential  time-lapse  experiments).  The  dish  will  be  removed  from  the  microscope  stage  to  the  incubator  for 
hours  or  days.  A^Tien  the  dish  returned  to  the  microscope  stage,  the  cells  may  still  be  alive,  or  they  may  be 
dead  (as  in  “fixed  and  stained”).  If  the  cells  are  dead,  time-lapse  experiments  are  not  performed.  However,  in 
this  case,  surrounding  fields  of  cells  will  also  be  imaged  (by  taking  “snapshot”  image  stacks)  to  verify  that  the 
single  set  of  cells  monitored  repeatedly  was  representative  of  many  surrounding  sets  of  cells. 

II.  Deflnitions: 

Channel:  The  channel  specifier  is  designated  in  our  filenames  by;  _0,  _1,  _2,  _3,  _4,  _5,  and  _X.  The  numbers  refer  to  a 
specific  position  on  the  optical  filter  wheel,  and  the  _X  refers  to  a  transmitted  light  image.  The  identity  of  the  optical 
filter  at  each  position  in  the  filter  wheel  is  noted  in  the  header  of  each  image  according  to  wavelength.  The  type  of 
transmitted  light  used  is  not  specified. 

Examples: 

dcl22999dc2.0001.l_0.ics;  This  image  was  acquired  with  the  filter  wheel  in  position  “0”.  If  one  looks  in  the 
image  header,  one  will  see  that  the  filter  at  this  position  is  a  360nm  fluorescence  excitation  filter. 

dcl22999dc2.0001.l_X.ics:  An  image  acquired  by  opening  the  transmitted  light  shutter.  This  image  may  be 
phase  contrast,  DIG,  bright  field,  etc. 

Notes: 

The  (maximum  wavelength)  identities  of  the  optical  filters  at  each  position  of  the  filter  wheel  are  contained  in 
the  editable  config  text  file  (below). 


Set  of  Cells:  All  the  cells  visible  in  a  particular  field  of  view  in  a  digital  image  at  a  specific  magnification.  Over  the  course 
of  hours  and  days,  single  living  cells  may  enlarge  and  divide  into  two  cells,  cells  may  disintegrate,  cells  may  die  and 
float  away,  and  individual  living  cells  may  also  move.  The  initial  location  of  a  set  of  cells  is  specified  as  a  location 
relative  to  a  (0,0)  point  which  is  a  “+”  (crosshairs)  symbol  physically  etched  on  the  surface  of  the  Petri  dish. 

Stack  of  images;  a  collection  of  all  the  specified  channels  at  a  single  time  point 

Example  :  The  7*  stack  in  a  time-lapse  series  of  image  stacks: 

dc 1 22999dc  1 3 .0007.  l_0.ics 

dc  1 22999dc  1 3 .0007. 1_1  .ics 

dcl22999dcl3.0007.l_2.ics 

dc  1 22999dc  1 3 .0007 . 1_3 .  ics 

dcl22999dc  13. 0007.1  X.ics 
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Recipe  File:  a  text  file  that  specifies: 

Which  fluorescence  channels  go  into  a  complete  image  stack 

Exposure  time  for  each  channel 

Order  in  which  image  channels  are  acquired 

Interval  between  acquired  image  stacks  in  time-lapse  experiments. 

Number  of  stacks  to  acquire. 

Identity  of  excitation  filters  at  each  location  in  the  filter  wheel 


III.  Iteration  Examples 


1 .  First  Iteration:  A  dish  of  cells  that  has  never  been  imaged. 

a.  Set  (0,0)  mark  on  the  computer  screen  using  crosshairs  etched  on  dish 

b.  Search  for  a  set  of  cells  to  observe  (various  shutters  will  need  to  be  opened  and  closed,  filter  wheel 
needs  to  move  to  different  positions,  and  program  must  keep  track  of  where  user  has  moved  relative  to 
the  (0,0)  mark). 

c.  Focus,  and  then  execute  the  experiment  based  on  information  in  the  recipe  file.  This  first  experiment 
will  usually  be  a  time-lapse  experiment  on  living  cells,  but  it  could  also  be  a  series  of  “snapshots”  of 
fixed  and  stained  cells. 

d.  Re-Edit  the  recipe  file  (DO  NOT  MOVE  CELLS  OR  STAGE.  REFOCUSING  IS  ALLOWED.)  and 
do  one  or  more  additional  time-laose  experiments. 

e.  Remove  cells  and  return  cells  to  incubator  OR,  (in  the  case  of  a  “snapshot”  experiment)  go  to  other 
locations  and  take  more  snapshots. 

2.  Second  Iteration;  A  set  of  living  cells  has  been  imaged  and  information  about  their  location  has  been  stored. 

a.  Manually  mount  and  position  the  dish  .  Acquire  an  initial  transmitted  light  image  of  the  crosshairs 
mark. 

b.  Click  on  the  (0,0)  mark  on  the  screen  and  tell  the  program  which  set  of  cells  these  are.  The  program 
will  move  to  the  (0,0)  mark  and  then  move  to  the  proper  field  of  cells. 

c.  Focus,  and  then  execute  the  experiment  based  on  information  in  the  edited  recipe  file,  (user  will  need 
to  open  shutters  and  move  the  filter  wheel  in  order  to  focus). 

d.  Re-Edit  the  recipe  file  (DO  NOT  MOVE  CELLS  OR  STAGE.  REFOCUSING  IS  ALLOWED.)  and 
do  one  or  more  additional  time-lapse  experiments. 

e.  Remove  cells  and  return  them  to  the  incubator  OR,  (in  the  case  of  a  snapshot  experiment)  go  to  other 
locations  and  take  more  snapshots. 

3.  Nth  Iteration:  A  set  of  living  cells  has  been  repeatedly  imaged  and  information  about  their  location  has 
been  stored.  Sequence  of  events  is  similar  to  second  iteration. 
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1  "A  Method  for  Knowledge-Based  Discovery  and  Optimization  of  Differences  Between  Cell  Types" 

DESCRIPTION: 

Background  of  the  invention  Please  summarize: 

1)  Technical  problems  overcome  to  make  the  invention, 

2)  What  your  invention  enables  people  to  do  that  couldn't  be  done  as  well  before, 

3)  How  people  currently  address  the  problem  your  invention  addresses. 

1.  (a)  Prior  to  this  invention,  it  was  technically  difficult  to  digitally  image  large 
numbers  of  single  living  cells  under  a  microscope.  It  was  also  difficult  to 
simultaneously  monitor  multiple  physiological  responses  in  large  numbers  of  single, 
living  cells  for  an  extended  period  of  time,  without  harming  the  cells. 

The  first  limitation  existed  because  of  the  limited  field  of  view  at  any  given  magnification.  The 
field  of  view  was  further  restricted  by  the  limited  size  and  resolution  of  the  solid-state  digital 
imaging  device  (charge  coupled  device)  inside  a  given  digital  camera.  This  invention  addresses 
these  technical  limitations  in  the  following  ways.(i)  It  rapidly,  automatically,  and  repetitively 
monitors  multiple  fields  of  view  (and  thus  larger  numbers  of  cells)  by  software  control  of  an  x,y,z 
microscope  stage  (ii)  It  allows  simultaneous  remote  control  of  multiple  microscopes  from  a 
central  computer. 

Living  cells  can  move  or  change  shape.  Thus,  when  returning  to  a  given  field  of  view,the 
software  must  adjust  focus  (z-axis  control),  and  must  make  small  horizontal  and  vertical 
adjustments  to  reacquire  and  reregister  the  field  of  view  (x,y  axis  control).  In  addition,  the 
original  contours  defining  the  outline  of  individual  cells  must  be  modified  in  order  to  track 
changes  in  cell  position  and  shape.  This  is  an  elementary  example  of  Visual  Servoing  Optical 
Microscopy  (VSOM),  where  decisions  regarding  control  of  the  instrument  are  made  based  on 
analysis  of  image  content. 

This  invention  allows  one  to  define  the  contours  of  living  cells  using  transmitted  lieht  in 
addition  to  fluorescence  emission  light.  This  allows  one  to  track  cells  that  do  not  contain  a 
fluorescent  compound.  In  addition,  this  allows  one  to  use  smaller  amounts  of  potentially 
C3h:otoxic  fluorescent  labeling  compounds  and  avoid  the  intense  fluorescence  excitation 
illumination  required  to  stimulate  fluorescence  emission.  Thus,  cells  can  be  observed  for  longer 
periods  of  time,  and  cells  with  specific  behaviors  can  be  retrieved  individually  at  the  end  of  the 
experiment  for  pooling  ,  establishment  of  cell  sublines,  cloning,  propagation  towards  a  given 
differentiated  state,  or  analysis. 

In  addition  to  monitoring  a  larger  number  of  cells,  this  invention  allows  a  larger  number  of 
physiological  responses  to  be  observed  during  a  single  experiment  because  the  instrument  can 
observe  different  physiological  responses  in  different  channels  (almost  simultaneously)  by 
rapidly  cycling  the  imaging  from  transmitted  light  to  fluorescence  emission  light  of  different 
wavelengths. 
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1.  (b)  A  technical  limitation  of  implementing  knowledge-based,  multi-channel, 
multi-field,  multi-microscope  Visual  Servoing  Optical  Microscopy  rVSOM)  was  the 
fact  that  the  method  is  memory  and  CPU-intensive,  and  the  proper  computer 
hardware  may  not  be  located  near  the  microscope  or  microscopes. 

Often,  the  local  computers  (the  clients)  that  control  local  microscope  peripherals  such  as 
scanning  stages,  filter  wheels,  perfusion  pumps,  robotic  arms,  shutters,  cameras,  etc.  will  not  have 
sufficient  memory,  software,  or  CPU  resources  to  perform  VSOM  rapidly  and  efficiently.  To 
overcome  this  limitiation,  our  method  allows  VSOM  to  be  performed  by  remote  control,  over  the 
Internet,  by  a  more  powerful  central  computer  (the  server)  located  in  a  different  geographical 
location.  The  server  can  perform  (or  distribute)  the  more  CPU  intensive  tasks  and  can  serve  as  a 
central  location  for  a  very  large  database  of  current  and  previous  cellular  responses. 

This  type  of  VSOM  benefits  greatly  from  network  access  to  a  powerful  central  server  and  a  large 
central  database  of  previously  observed  single  cell  responses  (with  previously  observed 
correlations  across  multiple  channels  of  information).  During  the  course  of  the  experiment,  the 
system  repeatedly  detects  cells,  logs  and  analyzes  all  the  observed  cell  responses  across  all 
channels,  compares  current  cell  responses,  makes  correlations  across  current  channels  of 
information  and  consults  a  database  of  previously  observed  cell  responses  and  previously 
observed  correlations  across  different  channels  of  information.  Then,  during  the  course  of  a 
single  experiment,  the  system  makes  a  series  of  knowledge-based  decisions  on  how  to  adjust 
experimetal  parameters  in  order  to  achieve  the  objective  of  the  current  experiment. 


2.  This  invention  allows  multiple,  complete  experimental  cycles  during  the  course  of 
a  single,  relatively  short  experiment.  This  greatly  accelerates  the  process  of 
searching  for,  discovering  and  optimizing  differences  between  different  cell  types. 
Relatively  low  numbers  of  cells  are  required,  and  because  individual  cell  responses 
are  monitored,  specimens  submitted  for  analysis  may  contain  several  different  types 
of  cells. 

For  large  numbers  of  individual  living  cells,  the  ability  to  rapidly  make  automated,  knowledge- 
based  decisions  on  how  to  alter  experimental  parameters  during  the  course  of  the  experiment  was 
not  available  prior  to  our  invention.  The  ability  to  perform  many  "test/  store/  analyze/  learn/ 
redesign/  retest"  cycles  during  the  course  of  a  single  experiment  will  allow  many  experiments  and 
applications  that  were  not  previously  feasible.  When  cells  remain  viable  and  an  experiment 
remains  in  progress  and  on-line,  correlations  between  past  and  future  responses  to  stimuli  are 
maintained  on  a  cell-by-cell  basis.  Various  and  repeated  computer-generated  stimuli  can  be 
applied  and  repeated  during  a  single  experiment  using  a  relatively  small  sample  of  cells.  This  is  a 
tremendous  advantage  when  searching  for  and  optimizing  differences  between  cell  types. 

3.  How  do  people  currently  address  the  problem  your  invention  addresses? 


Currently,  large  numbers  of  single  cell  responses  are  obtained  taking  a  single  measurement,  or 
"snap-shot"  of  cell  status,  response,  or  identity  at  a  single  instant.  For  microscope  studies,  cells 
are  often  fixed  (killed  and  preserved)  and  stained.  A  large  number  of  cells  can  then  be  scanned 
and  multi-charmel  fluorescence  and  transmitted  light  images  can  be  obtained  and  analyzed.  The 
cells  are  dead  and  cannot  be  repeatedly  stimulated.  In  microscope  studies  of  living  cells,  only  a 
limited  number  of  cells  (a  single  field)  can  be  observed  during  stimulation.  However,  the  images 


Annual  Report:  Callahan,  Parvin  Page  70 


3  "A  Method  for  Knowledge-Based  Discovery  and  Optimization  of  Differences  Between  Cell  Types" 


are  usually  analyzed  (often  days)  after  the  experiment.  Thus,  it  is  not  possible  to  make  rapid, 
automated  decisions  on  how  to  alter  experimental  parameters  during  the  course  of  the 
experiment. 

Ongoing  fluoresence  signals  from  an  entire  population  of  living  cells  can  be  detected  using  a 
multi-well  fluorescence  plate  reader.  Single  cell  responses  cannot  be  observed.  One  potential 
use  of  VSOM  is  the  discovery  and  optimization  of  fluorescence  assays  that  are  suitable  for  multi- 
well  plate  readers.  Multi-channel  flourescence  signals  from  individual  living  cells  can  be 
detected  at  a  single  instant  using  flow  cytometry.  The  cells  can  be  sorted  according  to 
fluorescence  signal  and  can  be  recovered.  However,  the  cells  flow  rapidly  past  a  detector,  and  a 
single  (perhaps  multi-channel)  measurement  is  taken.  Individual  cells  are  not  tracked  for  any 
length  of  time,  and  repeated  observations  of  the  same  cell,  with  correlations  between  past  and 
present  responses,  is  not  possible.  However,  another  potential  use  of  VSOM  is  the  discovery  and 
optimization  of  fluorescence  assays  that  are  suitable  for  flow  cytometry  and  fluorescence 
activated  cell  sorting. 


List  uses  of  the  invention  -  research,  commercial,  pilot  plant  etc.;  Think  as  broadly 
as  possible. 

1)  Fluoresence  Assays:  As  discussed  above,  many  instruments  currently  exist  that  can  take 
advantage  of  new,  cell-type  specific  fluorescence  assays.  These  instruments  and  assays  are 
currently  in  use  in  hospitals  and  other  diagnostic  laboratories.  These  assays  are  useful  in  human, 
animal,  and  plant  cell  studies.  The  invention  described  here  will  also  benefit  from  any  newly 
discovered  fluoresence  assays. 

Examples:  (a)  The  detection  and  discrimination  between  normal,  premalignant,  malignant,  or 
multi-drug  resistant  cancer  cells  derived  from  a  human  tissue  biopsy,  (b)  The  design  of  protocols 
that  selectively  label  rare  cell  types  such  as  stem  cells,  or  fetal  cells  in  maternal  blood.  Such 
protocols  could  be  modified  to  work  with  radioactive  labels,  also,  (c)  The  design  and  application 
of  assays  that  suggest  a  chemotherapeutic  regimen  ,  or  combination  of  drugs,  that  are  tailored  to 
an  individual  patient  or  an  individual  tumor  within  a  patient,  (d)  Assays  useful  for  screening 
large  numbers  of  potential  drug,  insecticide,  or  herbicide  candidates,  (e)  Assays  useful  for 
screening  large  numbers  of  potential  agents  useful  for  cell  proliferation,  cjdotoxicity,  or 
differentiation,  (f)  other  cell-type  specific  fluorescence  diagnostic  tests. 


2)  Design  of  in  vitro  tissue  culture  conditions  that  are  cell-type  specific 

With  development  of  suitable  fluorescence  assays,  this  invention  can  be  used  to  design  in  vitro 
tissue  culture  conditions  that  lead  to  the  proliferation  of  a  specific  cell  type,  either  by  giving  a 
growth  advantage  to  this  cell  type,  or  by  designing  environmental  conditions  or  protocols  that  are 
cytotoxic  to  other  cell  types  present  in  the  culture.  For  example,  it  is  often  difficult  to  progate 
tumor  cells  from  patient  biopsies  in  culture  so  that  the  characteristics  of  that  patient's  tumor  cells 
can  be  studied.  Normal  cells  often  grow  more  easily  in  vitro  than  tumor  cells.  Other  types  of 
cells  are  difficult  to  grow  in  culture,  and  this  invention  would  allow  one  to  rapidly  develop 
suitable  environmental  conditions  or  growth  maintenance  protocols  for  the  specific  cell  types  of 
interest.  In  addition,  cell  culture  conditions  suitable  for  guiding  cells  to  a  specific  differentiated 
end-point  could  be  designed  using  this  invention.  Thus,  the  conditions  necessary  to  guide  stem 
cells  or  embryonic  cells  to  a  final  desired  end  point  could  be  developed  using  this  invention. 
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3)  Design  and  application  of  other  in  vitro  diagnostic  tests 

With  development  of  suitable  fluorescence  assays,  this  invention  could  be  used  to  develop  and 
execute  many  types  of  rapid  in  vitro  tests.  For  example,  if  severall  potential  tissue  or  bone 
marrow  donors  existed  for  a  certain  patient,  the  various  donor's  cells  could  be  mixed  with  the 
patient's  immune  cells  and  observations  could  be  made,  on  the  single  cell  level,  of  #he  immune 
rejection  response  generated  by  each  potential  donor's  cells.  In  addition,  these  types  of  tests 
could  be  performed  in  the  presence  of  drugs  (or  drug  candidates)  designed  to  suppress  transplant 
rejection.  Such  tests  could  allow  one  to  choose  the  proper  donor  for  a  patient,  and  the  proper 
anti-rejection  drug  or  drugs  for  the  patient 

Identify  companies  that  might  be  interested  in  licensing  this  technology. 

Companies  that  provide  scientific  instruments  for  medical  or  scientific  applications  would  be 
interested  in  this  technology.  Especially  companies  specializing  in  cytometry  and  diagnostic 
applications;  these  companies  produce  digital  imaging  microscope  systems,  flow  C3dometers,  and 
other  sophisticated  medical  and  scientific  instrumentation. 


Summary  of  the  Invention 


We  have  developed  methods,  sofware  and  a  process  to  discover,  develop,  and  optimize 
fluorescence  assays,  protocols,  and  in  vitro  tissue  culture  conditions  that  are  based  on  differences 
between  cell  types.  This  approach  does  not  require  pure  populations  of  cells.  It  is  well  suited  for 
relatively  small  samples  and  heterogenous  mixtures  of  cells.  "Cell  type"  is  a  classification 
achieved  by  any  observation  that  can  separate  a  group  of  cells  into  multiple  groups  of  cells,  where 
each  cell  of  a  group  has  a  similar  property(ies)  that  distinguishs  it  from  another  group(s)  of  cells. 
The  method  requires  repeated  observations  of  relatively  rapid  physiological  responses  at  the 
individual,  single-cell  level  (including  observations  of  subcellular  components,  changes  in 
morphology,  cell  division,  cell  death,  detachment,  etc.).  The  term  "stimulus"  is  meant  to  include: 
a  single,  repeated,  or  continuous  application  of  a  stimulus,  stimuli,  or  cominations  of  stimuli. 
Stimuli  that  generate  physiological  responses  can  be  mechanical,  physical,  chemical,  or  biological 
in  nature.  One  distinguishing  features  of  this  method  is  rapid,  on-line  analysis  of  large  numbers 
of  living  cells  and  large  numbers  of  correlated  physiological  responses  (including  references  to  a 
database  of  responses,  and  correlation  across  multiple  channels  of  image  information  representing 
different  imaging  modalities  such  as  transmitted  light  and  fluoresence  emissions  at  a  variety  of 
excitation/emission  wavelengths).  Another  distinguishing  feature  of  this  method  is  computer 
automation  of  the  following:  (1)  repeated  or  continuous  application  of  the  stimulus  to  living  cells 
(2)  repeated  or  continuous  detection  and  monitoring  of  one  or  more  cellular  or  subcellular 
physiological  responses  to  the  stimulus  at  the  single  cell  level  using  a  microscope  and  digital 
imaging,  (3)  rapid  observation  of  a  sufficient  number  of  individual  cell  responses  to  establish 
criteria  suitable  for  (i)  reliable  identification  of  cell  type(s),  and/or  (ii)  modification  of  the 
stimulus  for  purposes  of  optimizing  differences  between  cell  types  (4)  rapid  analysis  of  current 
and  previously  stored  (as  in  a  database)  cell  responses  so  that  knowledge-based  stimulus  control 
decisions  can  be  made  while  current  cells  are  still  available  for  observation  and/or  stimulation. 
Correlation  of  time-dependent  events  across  multiple  channels  are  also  monitored. 

We  refer  to  the  process  described  above  as  Visual  Servoing  Optical  Microscopy  (VSOM).  In  its 
simplest  form,  VSOM  uses  computer  vision  to  analyze  a  single  field  of  view  under  a  microscope 


/ 


Annual  Report:  Callahan,  Parvin  Page  72 


5  "A  Method  for  Knowledge-Based  Discovery  and  Optimization  of  Differences  Between  Cell  Types" 


and  alter  microscope  peripherals  until  a  desired  goal  is  achieved.  A  trivial  example  is  that  of 
autofocusing.  The  computer  moves  a  microscope  objective  up  or  down  until  a  specified  degree  of 
sharpness  is  achieved  in  a  digital  image. 

Biological  systems  are  extremely  complex  and  often  heterogeneous  in  nature.  Cells  are 
continously  processing  and  responding  to  a  multitude  of  intracellular  and  extracellular  signals. 
Cells  rapidly  initiate  responses  to  subtle  changes  in  their  environment;  thus,  a  largd^' number  of 
experimental  variables  must  be  accounted  for.  In  addition,  individual  cells  of  the  same  cell  type 
exhibit  a  range  of  responses  to  the  same  stimulus.  Thus,  a  large  number  of  individual  cells  must 
be  observed  in  order  to  define  the  distribution  of  responses  that  are  possible.  Current  biolpgital 
instrumentation  (e.g.  flow  cytometiy)  makes  a  single  (possibly  multichannel)  measurent  otf  large 
numbers  of  cells  as  they  flow  past  a  detector.  However,  repeated  measurements  on  the  same  cell 
are  not  possible,  and  an  image  of  each  cell  is  not  obtained. 

Our  process  relies  on  computer  software  and  commercially  available  instrumentation  to  acliieve 
the  following: 

(i)  Detection  and  tracking  of  one  or  more  (usually  large  numbers)  individual  (usually  living)  cells 
in  one  or  more  fields  of  view.  This  detection  and  tracking  occurs  by  repeatedly  obtaining  one  or 
more  channels  of  digital  image  information  simultaneously  (or  in  rapid  succession).  Digital 
image  information  in  these  channels  can  consist  of  various  combinations  of  information  derived 
from  any  microscope  imaging  modality  (e.g.  transmitted  light  techniques  such  as  phase  contrast 
or  differential  interference  contrast,  fluorescence  excitation/emission  techniques,  three- 
dimensional  imaging,  etc.) 

(ii)  Identification  of  different  cell  types  or  subpopulations  of  cell  types. 

(iii)  Discovery  and/or  optimization  of  cell-type-specific  physiological  responses. 

(iv)  Knowledge-based  control  of  the  microscope  and  microscope  peripherals  to  modify  stimuli 
and  selectively  optimize  the  unique  physiological  response(s)  of  specific  cell  types. 

(v)  Allow  retrieval  of  specific  individual  cells  at  the  end  of  the  experiment. 


Publications  (including  LBL  reports,  meeting  abstracts,  or  prior  patents  or  patent 
applications)  that  ACTUALLY  DESCRIBE  or  RELATE  to  the  invention: 


NOTE;  No  publications  actually  describe  this  invention. 

These  publications  relate  to  invention: 

To  be  presented: 

Parvin,  B.,  Taylor,  J.R.,  Cong,  G.,  O'Keefe,  M.,  Barcellos-Hoff  "Deep  View:  A  Channel  for 
Distributed  Microscopy  and  Informatics"  IEEE  Conference  on  High  Performance  Computing  and 
Networking"  November  1999,  Portland,  OR. 
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Parvin,  B.,  Taylor,  J.R.,  Cong,  G.  "Deep  View:  A  Channel  for  Distributed  Microscopy" 
International  Conference  on  Distributed  Objects,  Scotland,  UK,  Early  September,  1999. 


Already  Presented  or  Published: 

Parvin,  B.,  Taylor,  J.R.,  Cong,  G.  "A  Collaborative  Framework  for  Distributed  N^icroscopy" 
IEEE  Conference  on  High  Performance  Computing  and  Networking"  November  1998. 

Parvin,  B.,  Taylor,  J.R.,  Callahan,  D.E.,  Johnston,  W.E.,  Dahmen,  U.  "Visual  Servoing  for  Online 
Facilities"  (1997)  IEEE  Computer  30 , 56-62. 

Parvin,  B.,  Callahan,  D.,  Johnston,  W.,  Maestre,  M.  "Visual  Servoing  for  Micro-Manipulation", 
13th  Int.  Conf.  on  Pattern  Recognition,.  Aug  25-30, 1996,  Vienna,  Austria. 

Parvin,  B.,  Callahan,  D.  Johnston,  W.,  Maestre,  M.  "Microdissection  of  Single  DNA  Molecules" 
Fifth  DOE  Human  Genome  Program  Contractor-Grantee  Workshop,  Santa  Fe,  NM,  Jan28-Feb  1 . 
(1996) 

Parvin,  B.,  Maestre,  M.F.,  Fish,  R.H.,  Johnston,  W.  "A  Method  and  Apparatus  for  Manipulation 
of  an  Object  on  a  Microscope  Stage"  US  Patent  Application  Filed  April,  1995.  IB-1025  and  IB- 
1052. 

Parvin,  B.,  Taylor,  B.  Crowley,  B.,  Wu,  L.,  Johnston,  W.,  Owen,  D.,  O'Keefe,  M.A.,  Dahmen,  U. 
"Telepresence  for  In-Situ  Microscopy"  IEEE  Conference  on  Multimedia  Systems  and  Computers, 
June  1996. 

Other  early  documentation  that  relates  to  the  invention: 

Callahan,  D.E.  and  Parvin,  B.  (Principal  Investigators),  "Visual  Servoing  for  Optimization  of 
AntiCancer  Drug  Uptake  in  Human  Breast  Cancer  Cells"  LBNL  proposal  submitted  to  DOD/US 
Army  Breast  Cancer  Research  Program,  Second  submission,  July  1997  (funded). 

Callahan,  D.E.  (Principal  Investigator), "Visual  Servoing  for  Optimization  of  AntiCancer  Drug 
Uptake  in  Human  Breast  Cancer  Cells"  LBNL  proposal  submitted  to  DOD/US  Army  Breast 
Cancer  Research  Program,  First  submission,  July  1996  (not  funded). 

Callahan,  D.E.  (Principal  Investigator)  "Calcium  Homeostasis  in  Normal  and  Transformed 
Mammary  Cells"  LBNL  proposal  submitted  to  California  Breast  Cancer  Research  Program, 
January  1996  (not  funded). 

Parvin,  B.,  (Principal  Investigator)  "Ordered  Micro  Dissection  and  Retrieval  of  DNA  Molecules" 
NIH  ROl  (1R01HG01443-01)  LBNL  proposal  submitted.  May  1995  (not  funded) 

Parvin,  B.,  Johnston,  W.,  Dahmen,  U.,  Callahan,  D.,  Maestre,  M.,  Liburdy,  R.  "Visual  Servoing 
for  Scientific  Applications"  LBNL  LDRD  #2  (Laboratory  Directed  Research  and  Development) 
proposal  submitted  April  30, 1994  (funded) 

Callahan,  D.E.,  Liburdy,  R.P.,  Maestre,  M.F.,  Johnston,  W.E.,  Parvin,  B.,  Elaydi,  T.S.  "Calcium 
Oscillations  in  Jurkot  Cells"  Internal  LBNL  Video  V-943-13,  April  1994. 
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Johnston,  W.,  Parvin,  B.,  Maestro,  M.,  Callahan,  D.  Liburdy,  R.,  Dahmen,  U.  "Visual  Servoing  in 
a  High  Speed  Network  Environment",  LBNL  LDRD  #1,  proposal  submitted  April  30,  1993 
(funded). 
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Technical  Abstract;  Rapid  Discovery  of  Physiological  Characteristics  Which  Distinguish  Malignant  and 
Nonmalignant  Human  Breast  Epithelial  Cells.  Callahan,  D.E.,  Parvin,  B.,  and  Dairkee,  S.H. 

Background:  Three  prospective  clinical  trials  have  attempted  to  use  in  vitro  chemosensitivity  testing  to 
improve  patient  chemotherapy  response  and  cancer  survival.  The  trials  failed  to  demonstrate  increased  survival 
for  patients  who  were  administered  chemotherapy  based  on  results  of  in  vitro  chemosensitivity  tests.  Two  in 
vitro  chemosensitivity  assays  were  used,  the  HTCA  (human  tumor  cloning  assay)  and  the  DiSC  (differentia 
staining  cytotoxicity)  assay.  In  these  two  assays,  the  in  vitro  cell  culture  and  propagation  techniques  used  and 
the  biological  endpoints  measured  differ  significantly.  A  common  set  of  problems  were  noted  in  the  data 
emanating  from  these  trials.  Both  assays  (1)  are  manual  and  labor  intensive  in  nature,  (2)  are  dependent  on 
human  judgment  for  quantitation,  (3)  often  fail  to  produce  any  results,  because  tumor  cells  from  some  patients 
do  not  grow  well  in  these  in  vitro  systems.  For  reasons  such  as  these,  clinician  perception  and  acceptance  ol 
these  assays  is  poor;  thus,  patient  and  specimen  accrual  has  been  difficult.  Our  group  has  the  expertise  tc 
address  these  problems  using  advanced  cell  culture  techniques  and  innovative  technology  derived  from  the 
fields  of  robotic  vision  and  digital  imaging  fluorescence  microscopy.  With  the  successful  completion  of  our 
proposed  studies,  we  expect  to  demonstrate  that  in  vitro  chemosensitivity  tests  based  on  a  single  protocol  or 
assay  will  rarely  be  successful.  We  will  use  the  clinical  observations  and  findings  that  have  emanated  from 
three  prospective  clinical  trials,  and  we  will  demonstrate  a  new  paradigm  for  the  scientific  design  of 
chemotherapeutic  regimens  tailored  to  the  individual  breast  cancer  patient. 

Hypothesis:  Our  hypothesis  is  that  previous  clinical  studies  that  sought  to  administer  anticancer  drugs  based 
on  in  vitro  chemosensitivity  tests  failed  to  see  dramatic  increases  in  patient  response  or  survival  for  several 
specific  reasons:  (i)  the  microenvironment  favored  by  tumor  cells  was  not  properly  simulated  in  vitro  (ii)  the 
physiological  responses  of  individual  cells  were  not  monitored  closely  throughout  the  assay,  and  (iii)  the  drug 
concentrations,  combinations,  and  exposure  times  examined  were  limited,  and  not  scientifically  optimized.  We 
further  hypothesize  that  Visual  Servoing  Optical  Microscopy  (VSOM)  can  rapidly  discover  and  exploit  key 
physiological  characteristics  which  distinguish  malignant  and  nonmalignant  human  breast  epithelial  cells 
(BECs).  In  this  way,  VSOM  will  help  to  solve  problems  (i-ii),  listed  above. 

Objective:  Demonstrate  and  verify  that  VSOM  can  rapidly  discover  and  optimize,  on  a  tumor-by-tumor 
basis,  microenvironmental  conditioas  suitable  for  the  successful  in  vitro  propagation  and  chemosensitivity 
testing  of  primary  human  breast  tumor  cells.  Our  specific  aims  are  to  (a)  rapidly  and  automatically  develop  in 
vitro  microenvironments  that  favor  the  propagation  of  primary  breast  tumor  cells,  (b)  rapidly  develop  new  in 
vitro  fluorescence  assays  that  predict  cell  behavior  on  the  basis  of  early  physiological  responses  to  applied  stress 
(c)  improve  VSOM  technology  in  order  to  achieve  our  objectives  and  specific  aims. 

Study  Design:  We  will  study  a  well  characterized  set  of  human  malignant  and  nonmalignant  BECs.  Using 
our  demonstrated  expertise  in  this  field,  we  will  propagate  tumor  cells  as  monolayers,  and  this  will  allow  us  to 
perform  a  wide  range  of  multicolor,  multichannel  VSOM  fluorescence  assays  in  a  temperature-controlled 
microperfusion  chamber.  The  intracellular  calcium  levels,  levels  of  mRNA,  and  intracellular  pH  of  individual 
cells  will  be  monitored  and  manipulated  during  VSOM  experiments.  Early  physiological  responses  to 
perturbations  of  the  microenvironment  (pH,  p02,  [Ca'*'^]ex  ,anticancer  drugs)  will  be  stored  in  our  bioinformatics 
database.  The  ultimate  effect  of  these  perturbations  will  be  assessed  by  using  VSOM  to  return  to  the  same  field 
of  cells  after  longer-term  incubations.  In  addition,  responses  that  appear  to  predict  an  increase  in  proliferation  or 
apoptosis  will  be  verified  in  multiwell  plate  fluorescence  assays  during  longer-term  incubations.  Proliferation, 
cell  viability,  and  apoptosis  will  be  monitored  using  VSOM  and  multiwell  plate  fluorescence  assays. 

Relevance:  Factors  responsible  for  the  previously  unsuccessful  application  of  in  vitro  chemosensitivity 
testing  in  the  clirtic  should  become  apparent  during  our  proposed  VSOM  experiments.  We  believe  VSOM 
technology  will  demonstrate  a  practical  solution  to  these  problems  and  this  will  necessitate  new  clinical  trials  at 
the  end  of  three  years.  We  believe  clinician  participation  and  patient  accrual  will  improve  if  we  can 
demonstrate  the  advantages  of  this  improved,  sophisticated  approach  to  in  vitro  chemosensitivity  testing. 
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Public  Abstract:  Rapid  Discovery  of  Physiological  Characteristics  Which  Distinguish  Malignant  and 
Nonmalignant  Human  Breast  Epithelial  Cells.  Callahan.  D.E.,  Parvin,  B.,  and  Dairkee,  S.H. 

In  three  clinical  trials  researchers  have  attempted  to  find  an  improved  method  for  selecting  the  best 
chemotherapy  drug  or  combination  of  drugs  for  their  patients.  Scientists  and  physicians  know  that  every  patient 
is  slightly  different;  for  example,  one  patient  may  have  an  adverse  reaction  to  a  drug,  vJ’hile  another  patient 
experiences  no  side  effects.  Doctors  often  deal  with  this  problem,  and  they  deal  with  it  using  their  training,  ■ 
experience,  and  a  trial-and-error  method  where  another  drug  is  substituted  for  the  one  that  causes  side  effects,  i 
In  the  case  of  breast  cancer,  doctors  must  choose  several  drugs  from  a  wide  range  of  possibilities  and  find  the] 
combination  that  (1)  is  most  effective  against  that  patient's  tumor,  and  (2)  produces  the  fewest  side  effects  in  the' 
patient.  Breast  tumor  cells  from  different  patients  can  be  very  different  and  respond  differently  to  the  same 
drug.  Thus,  for  over  40  years,  attempts  have  been  made  to  remove  tumor  cells  from  the  patient's  body  and  test 
them  "in  vitro",  which  means  outside  the  living  body,  in  an  artificial  environment.  This  same  concept  is  often 
applied  to  the  living  breast  cancer  patient.  For  example,  the  physician  may  prescribe  a  chemotherapy  drug  and 
find  that  a  patient's  tumor  does  not  respond,  or  shrink.  The  physician  then  tries  another  drug  for  this  patient, 
and  then  perhaps  another.  These  are  "in  vivo"  drug  response  tests,  meaning  they  are  performed  in  a  living  body. 
Unfortunately,  this  can  be  very  debilitating  to  the  patient,  and  there  are  just  too  many  drugs  and  too  many 
possible  combinations  to  make  this  approach  work.  The  potential  advantage  of  testing  the  drug  response  of 
cells  outside  the  patient's  body  is  that  a  wider  range  of  drugs  can  be  tested,  and  the  patient  is  spared  a  series  of 
debilitating  trial-and-error  attempts.  However,  due  to  the  reported  shortcomings  of  the  "first  generation"  of  in 
vitro  drug  response  tests,  the  entire  idea  of  in  vitro  drug  response  testing  has  fallen  out  of  favor  with  practicing 
clinicians.  This  has  occurred  to  such  an  extent  that  researchers  have  trouble  convincing  clinicians  to  refer 
patients  to  clinical  trials  where  "second  generation"  in  vitro  drug  tests  are  being  performed.  Due  to  lack  of 
support  and  lack  of  funding,  many  leaders  in  the  field  have  left  academia  to  continue  their  research  at  private 
companies,  many  located  in  Southern  California  (http://www.oncodocs.com  /laboratories/  laboratories.html; 
See  also,  "Pretesting  Tumors",  Scientific  American.  February,  1999,  pp.  19-22.).  Other  pioneers  in  this  field  of 
research,  who  remain  in  academia,  such  as  Dr.  D.D.  Von  Hoff,  currently  the  President  of  the  AACR  (American 
Association  for  Cancer  Research),  have  commented  that,  "It  is  indeed  a  puzzle  as  to  why  in  vitro  predictive 
assays  have  not  been  integrated  at  least  into  some  aspect  of  the  care  of  patients  with  cancer."  (Commentary, 
Journal  of  the  National  Cancer  Institute.  82,  1990,  pp.  96-101).  This  puzzle  exists  because  the  second 
generation  tests  have  proven  to  be  excellent  at  discovering  which  drugs  have  no  effect  on  a  patient's  tumor  cells. 
Thus,  doctors  do  not  have  to  consider  these  drugs  when  performing  in  vivo  drug  response  tests  on  patients. 
This  is  indeed  a  great  accomplishment  considering  the  fact  that  the  technology  used  in  these  second  generation 
tests  is  no  more  advanced  that  what  was  available  in  the  1950s.  These  tests  are  performed  manually,  with  a 
technician  counting  cells  or  colonies  under  a  microscope.  Thus,  the  tests  are  slow,  labor  intensive  and  subject 
to  human  error.  In  our  proposed  research  we  describe  how  we  will  apply  21st  century  technology  to  this 
important  problem,  thus  creating  a  "third  generation"  of  in  vitro  drug  response  tests.  Previous  clinical  trials 
have  provided  us  with  a  great  deal  of  data  that  we  will  use  as  we  improve  these  in  vitro  tests.  We  believe  our 
new  automated  instrument,  a  Visual  Servoing  Optical  Microscopy  (VSOM)  will  be  able  to  overcome  the 
problems  reported  in  the  first  set  of  clinical  trials.  These  problems  include  (1)  the  difficulty  in  getting  tumor 
cells  to  grow  in  vitro  at  all  (2)  the  limited  ability  to  observe  cells  at  intervals  during  the  course  of  the  assay  (3) 
the  limited  number  of  biological  endpoints  that  could  be  observed  per  assay  (4)  the  lack  of  automation.  Our 
studies  will  demonstrate  and  verify  that  VSOM  can  rapidly  discover  and  optimize,  on  a  tumor-by-tumor  basis, 
microenvironmental  conditions  suitable  for  the  successful  in  vitro  propagation  and  chemosensitivity  testing  of 
primary  human  bijeast  tumor  cells.  Our  specific  aims  are  to  (a)  rapidly  and  automatically  develop  in  vitro 
microenvironments  that  favor  the  propagation  of  primary  breast  tumor  cells,  (b)  rapidly  develop  new  in  vitro 
fluorescence  assays  that  predict  cell  behavior  on  the  basis  of  early  physiological  responses  to  applied  stress  (c) 
improve  VSOM  technology  in  order  to  achieve  our  objectives  and  specific  aims.  We  believe  that  successful 
completion  of  these  goals  will  lead  to  new  clinical  trials  that  will  demonstrate  the  ability  of  VSOM  to  predict 
patient-specific  responses  and  establish  a  scientific  basis  for  chemotherapy  treatment  planning. 
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Relevance  and  Impact  Statement 

For  some  types  of  cancer  such  as  certain  leukemias  and  testicular  cancer,  chemotherapy  can  cure  a  majority 
of  patients.  This  demonstrates  the  potential  of  chemotherapy  to  effect  a  cure  in  some  instances  of  cancer. 
However,  in  solid  tumors  such  as  breast,  little  progress  has  been  made  in  improving  therapy.  Multidrug 
resistance  (MDR),  inherent  or  acquired,  in  solid  tumors  such  as  breast  has  been  cited  as  onfe  important  obstacle 
preventing  cures  via  chemotherapy.  In  vitro  chemosensitivity  assays  to  assess  drug  response  and  predict  patient 
response  have  been  in  development  for  over  forty  years  but  a  truly  successful  in  vitro  chemosensitivity  test  has 
not  been  developed.  Our  approach  to  in  vitro  drug  response  and  chemosensitivity  testing  represents  a  radically 
different  approach  and  a  new  paradigm.  We  believe  our  improvements  in  in  vitro  drug  response  and 
chemosensitivity  assays  will  allow  patient-directed  therapies  that  improve  clinical  outcomes  for  breast  cancer  i 
patients.  Digital  imaging  fluorescence  microscopy,  real-time  analysis  of  digital  images,  and  bioinformatics 
databases  have  evolved  to  the  point  where  a  new  class  of  experiments  is  possible.  Visual  servoing  (VS),  a 
robotic  vision  technique,  refers  to  the  dynamic  manipulation  of  experimental  parameters  based  on  analysis  of 
digital  image  content.  In  addition,  a  visual  servoing  optical  microscope  (VSOM)  has  the  ability  to  scan  back 
and  forth  across  multiple  fields  of  view  and  in  this  way  monitor  very  large  numbers  or  different  populations  of 
living  cells.  In  addition,  VSOM  can  also  return  repeatedly  to  the  same  field  of  cells  to  monitor  changes  on  a 
cell  by  cell  basis.  This  capability  will  help  us  achieve  our  objective  of  making  correlations  between  the  early 
physiological  responses  observed  in  cells  and  the  future  biological  state  or  ultimate  biological  fate  of  the  cells, 
whether  this  be  proliferation,  apoptosis,  or  arrest  in  some  stage  of  the  cell  cycle.  In  our  opinion,  the  most 
desirable  predictive  assays  are  dynamic  and  adaptive  VSOM  fluorescence  assays  involving  the  observation  of  a 
series  early  physiological  response  to  intelligently  chosen,  computer-controlled  stimuli.  This  approach  becomes 
more  powerful  when  such  stimuli  and  perturbations  are  reversible  and  do  not  permanently  commit  the  cell  to  a 
given  biological  path.  The  ability  to  repeatedly  apply  relatively  innocuous  stimuli  and  perturbations  to  living 
cells  will  allow  us  to  repeatedly  probe  the  cell  for  physiological  characteristics  that  can  be  exploited  by 
anticancer  drugs  or  combinations  of  anticancer  drugs.  Our  creation  of  a  modem  object-oriented  database 
containing  records  of  previously  observed  physiological  cell  responses  will  become  very  valuable  in  the  future 
when  we  implement  knowledge-based  control  of  VSOM  assays.  The  ability  to  access  such  a  central  database 
remotely  from  any  VS-enabled  microscope  will  make  these  instmments  controllable  over  the  Internet  and  will 
greatly  increase  their  functionality  while  greatly  decreasing  the  expense  of  these  instruments  and  the  computer 
power  required  to  operate  them.  This  will  be  an  advantage  for  VSOMs  located  in  clinics  lacking  the  resources 
to  build  and  maintain  such  a  database.  Currently,  we  believe  that  individualized  patient  treatment  and  patient- 
specific  analysis  of  tumor  cells  begins  with  characterizing  and  then  recreating  the  specific  microenvironmental 
conditions  necessary  to  propagate  and  amplify  the  cells  for  in  vitro  testing.  Once  this  is  achieved,  a  variety  of 
anticancer  drugs,  and  dmg  combinations  can  be  tested  using  standard  multiwell  plates  and  fluorescence  assays. 
Thus,  these  are  the  intitial  VSOM  experiments  we  will  perform.  We  believe  this  proposal  meets  the  stated 
intent  of  the  Bridge  Award  mechanism.  It  is  likely  that  the  successful  demonstration  of  our  technology  will 
result  in  new,  more  successful  clinical  trials.  Thus,  the  proposed  research  can  be  considered  proximal  to  clinical 
trials.  In  addition,  it  may  be  considered  a  Post-Clinical  Trial  Follow-Up  Study  in  the  sense  that  it  that  uses  the 
clinical  observations  and  findings  that  emanated  directly  from  three  previous  prospective  clinical  trials.  While 
we  will  not  use  the  exact  tumor  specimens  used  in  the  previous  studies,  we  will  use  a  well  characterized  set  of 
malignant  breast  epithelial  cells  (BECs)  derived  from  a  range  of  human  breast  cancer  patients.  Nonmalignant 
BCEs  will  also  be  studied.  We  will  be  able  to  determine  whether  the  stated  microenvironmental  conditions 
employed  in  previous  clinical  trials  (composition  of  the  media,  pH,  pOa,  etc.)  need  to  be  altered  for  successful 
propagation  and  testing  of  primary  tumor  cells  in  culture,  and  we  will  determine  the  range  of  conditions  favored 
by  breast  tumor  cells  from  multiple  individuals.  In  addition,  we  will  determine  whether  other  factors  such  as 
the  drug  concentrations  and  exposure  times  used  in  previous  studies  were  optimal,  and  we  will  determine  if  any 
or  all  of  such  factors  contributed  to  fact  that  previously  clinical  trials  have  reported  little  or  no  improvement  in 
patient  survival  when  chemotherapy  was  selected  on  the  basis  of  in  vitro  drug-sensitivity  tests. 
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Specific  Aim  1: 

Task  1  (months  1-12); 


Task  2  (months  13-30): 


Task  3  (months  31-36): 

Specific  Aim  2: 

Task  4  (months  12-24): 


Rapidly  and  automatically  develop  in  vitro  microenvironments  that  favor  the 
propagation  of  primary  breast  tumor  cells 

Begin  VSOM  experiments  using  very  small  samples  of  mali’gnant  BECs  derived 
from  human  patients.  Initial  experiments  will  examine  the  tumor  cells  of  lOj 
individuals  Deposit  small  numbers  of  these  cells  (2000-3000  cells)  onto  welli 
defined  circular  regions  of  cell  culture  dishes  using  cell  sedimentation  chambers.! 
Monitor  intracellular  calcium  levels,  intracellular  pH,  mitochondrial  membrane! 
potential,  mRNA  levels,  and  BrdU  incorporation  in  individual  living  cells  as' 
various  growth  stimuli  and  environmental  stresses  are  applied.  Observe  early  (8- 
12h)  cell  responses,  then  return  cells  to  a  controlled  environment  for  longer 
experiments.  Return  to  the  same  cells  later  (24-96h)  and  make  correlations 
between  initial  observed  responses  and  important  biological  endpoints.  Do  this 
on  a  cell-by-cell  basis.  Find  conditions  that  result  in  proliferation  and  then 
amplify  cells. 

Using  the  amplified  cells  from  Task  1,  perform  long-term  incubation,  multiwell 
plate  surveys  of  nonmalignant  and  malignant  BECs  in  order  to  verify  the 
microenvironmental  results  obtained  during  VSOM  experiments.  If  necessary, 
place  special  sealed  gas  chambers  inside  incubators,  and  perfuse  the  chamber  with 
gas  mixtures  of  known  composition.  The  pO?  and  pCOa  will  be  held  constant  in 
each  gas  chamber  in  order  to  generate  a  given  p02  or  pH  in  all  the  wells  of  all  the 
plates  contained  in  a  chamber.  Other  experimental  parameters  will  be  varied 
across  the  wells  of  the  plates  such  as  [Ca‘*'^]ex,  glucose,  anticancer  drug 
concentration,  and  concentration  of  important  growth  stimulants  such  as 
thapsigargin.  In  addition  to  variations  in  the  concentrations  of  these  agents,  the 
duration  of  exposure  will  also  be  varied.  Plates  will  be  removed  at  specified 
intervals  and  read  on  a  fluorescence  plate  reader.  Fluorescence  assays  for  cell 
number,  live/dead  ratio,  apoptosis,  and  proliferation  index  will  be  performed. 
Data  resulting  from  this  wider  sampling  of  combinations  of  environmental 
stresses  will  also  be  integrated  into  our  bioinformatics  database. 

Using  conditions  selected  from  Task  2,  perform  a  limited  number  of  clonogenic 
assays  in  soft  agar  and  determine  whether  it  is  important  to  conduct  in  vitro  drug 
response  assays  under  carefully  controlled  microenvironmental  conditions  that 
closely  reflect  the  in  vivo  tumor  cell  microenvironment . 

Rapidly  develop  new  in  vitro  fluorescence  assays  that  predict  cell  behavior 
on  the  basis  of  early  physiological  responses  to  applied  stress 

Perform  dynamic  VSOM  experiments  where  controlled  modulation  of 
intracellular  calcium,  intracellular  pH,  and  mitochondrial  membrane  potential  is 
performed  after  exposure  of  cells  to  specific  pharmacological  agents.  This  is 
achieved  by  using  computer-controlled  perfusion  pumps  to  perform  buffer 
exchanges  at  specified  intervals  after  exposure  of  the  cells  to  specific 
pharmacological  agents.  These  are  commonly  referred  to  as  "clamping" 
experiments.  In  this  way,  the  intracellular  concentrations  of  specific  ions  can  be 
clamped,  or  coupled  to  the  extracellular  concentration  of  ions,  an  experimental 
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parameter  that  is  under  direct  computer  control.  Use  this  ability  to  search  for  cell- 
specitlc  differences  that  can  be  exploited  for  identification  purposes,  or  as  a 
means  to  selectively  perturb  a  specific  subpopulation  of  cells.  In  addition,  use 
this  ability  to  more  precisely  relate  intracellular  conditions  to  eventual  biological 
endpoints  such  as  proliferation  and  apoptosis. 

i 

Use  MF-VSOM  to  compare  and  contrast  physically  separated  patches  of  cells  in 
co-culture.  One  patch  of  cells  will  contain  relatively  pure  tumor-derived  BEC  and 
the  second  patch  of  cells  will  be  derived  from  matched  nonmalignant  peripheral 
tissue.  By  moving  back  and  forth  between  the  two  groups  of  cells,  MF-VSOM 
will  automatically  search  for  very  subtle  differential  responses  to  physiological 
stresses  that  it  applies  automatically.  Knowledge  for  this  system  control- will  be 
partially  derived  from  the  bioinformatics  database.  Using  this  technique,  develop 
improved  VSOM  fluorescence  assays  so  that  it  is  possible  to  correctly  identify 
and  monitor  specific  subpopulations  of  cells  within  complicated  mixtures  of  cells. 
An  initial  matched  set  of  cells  representing  10  individuals  will  be  used. 

Improve  VSOM  technology  in  order  to  achieve  our  objectives  and  specific 
aims. 

Perfect  the  VSOM  ability  to  return  to  the  same  field  of  cells  using  a  variety  of 
vessels,  such  as  dishes,  flasks,  and  multiwell  plates.  Allow  VSOM  to 
automatically  apply  and  detect  responses  to  alterations  in  the  biochemical  milieu 
of  cells  during  on-line  VSOM  experiments.  Populate  the  bioinformatics  database 
with  single  cell  responses  and  search  for  correlations  between  observed  responses 
and  biological  endpoints.  Perform  this  off-line  data  mining  in  order  to  discover 
and  optimize  early  physiological  responses  (and  the  relevant  fluorescence  assays) 
that  have  a  high  predictive  value  for  specific  biological  endpoints. 

Develop  and  perfect  multi-field  (MF)  VSOM. 

Develop  knowledge-based  algorithms  for  VSOM  system  control  that  take 
advantage  of  previously  observed  cell  responses  stored  in  our  bioinformatics 
database. 
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Introduction:  Recent  advances  in  the  in  vitro  propagation  of  primary  breast  tumor  cells  allow  relatively 
small  numbers  of  tumor  cells  harvested  from  fine-needle  aspirates  (FNAs)  to  be  passaged  and  expanded  in 
culture  (Li  et  al.,  1998).  Moreover,  the  resultant  cells  in  culture  (up  to  10^  cells)  closely  resemble  the  original 
tumor  and  display  one  or  more  tumor  phenotypes,  including  growth  in  soft  agar.  We  bi/lieve  the  culture  of 
primary  breast  tumor  specimens  can  be  further  improved  through  the  use  of  our  new  technology  which  is 
derived  from  the  fields  of  robotic  vision,  fluorescence  microscopy,  biophysics,  and  bioinformatics.  Visual 
servoing  (VS)  is  a  term  from  the  field  of  robotic  vision.  In  the  proposed  studies,  VS  refers  to  the  dynamic 
manipulation  of  microenvironmental  parameters  based  on  analysis  of  digital  image  content.  Our  goal  in  the 
proposed  research  is  to  use  visual  servoing  optical  microscopy  (VSOM)  to  rapidly  characterize  large 
populations  of  living  cells  that  have  been  freshly  derived  from  human  breast  cancer  tissue  biopsies  or  fine 
needle  aspirates  (FNAs).  We  believe  VSOM  chemosensitivity  assays  will  allow  critical  information  for 
individualized  cancer  chemotherapy  to  be  obtained  in  a  rapid,  automated  fashion.  It  is  important  to  note  that  in 
vitro  chemosensitivity  testing  originated  in  the  1950s,  and  a  majority  of  the  previous  work  on  the  development 
of  in  vitro  chemosensitivity  assays  occurred  in  the  late  1970s  and  early  1980s.  This  was  prior  to  the  revolution 
in  computing  power,  digital  imaging  fluorescence  microscopy,  and  robotic  vision.  The  advances  in  these  fields 
over  the  past  twenty  years  have  been  impressive,  and  allow  for  the  development  and  application  of  a  new 
paradigm  in  the  field  of  in  vitro  chemosensitivity  testing.  This  new  paradigm  is  based  on  visual  servoing  and 
adaptive,  computer-controlled  testing  of  individual,  living  cancer  cells  derived  from  the  individual  breast  cancer 
patient. 

We  believe  this  proposal  meets  the  stated  intent  of  the  Bridge  Award  mechanism.  It  is  very  likely  that  the 
successful  demonstration  of  our  technology  and  our  new  paradigm  will  make  it  imperative  that  new  clinical 
trials  be  conducted  in  order  to  reevaluate  the  efficacy  of  in  vitro  chemosensitivity  testing  and  other  predictive 
assays  for  the  individual  breast  cancer  patient.  Thus,  the  proposed  research  can  be  considered  proximal  to 
clinical  trials.  In  addition,  it  may  be  considered  a  Post-Clinical  Trial  Follow-Up  Study  in  the  sense  that  it  that 
uses  the  clinical  observations  and  findings  that  emanated  directly  from  three  previous  prospective  clinical  trials 
(Gazdar  et  al.,  1990;  Von  Hoff  et  al.,  1990;  Wilbur  et  al.,  1992).  We  are  proposing  novel  techology  and 
protocols  that  will  improve  and  expand  upon  assays  that  have  previously  been  in  clinical  trials  (Weisenthal  and 
Lippman,  1985).  Many  clinical  observations  and  findings  have  emanated  from  these  and  other  clinical  trials,  and 
our  investigations  will  lead  do  a  new  understanding  of  these  clinical  data  and  to  the  development  of  new  clinical 
hypotheses.  In  addition,  our  technology  has  high  translational  potential.  For  example,  we  have  recently  filed  a 
provisional  patent  application  entitled,  "A  Method  for  Knowledge-Based  Discovery  and  Optimization  of 
Differences  Between  Cell  Types"  (LBNL  Patent  Docket  #5-94,163).  It  should  be  noted  that  several  companies, 
located  in  Southern  California,  have  recently  formed  for  the  purpose  of  conducting  such  in  vitro 
chemosensitivity  testing,  also  referred  to  as  predictive  testing  (Strauss,  1999  February). 

While  we  will  not  use  the  exact  tumor  specimens  used  in  the  previous  clinical  studies,  we  will  use  a  set  of 
well  characterized  malignant  breast  epithelial  cells  (BECs)  derived  from  a  range  of  human  breast  cancer 
patients.  Nonmalignant  BCEs  will  also  be  studied.  We  will  be  able  to  compare  our  results  with  data  and  results 
generated  by  the  previous  trials.  For  example,  we  will  be  able  to  determine  whether  the  stated  or  implied 
microenvironmental  conditions  employed  in  previous  clinical  trials  (composition  of  the  media,  pH,  p02,  etc.) 
need  to  be  altered  for  successful  propagation  and  testing  of  primary  tumor  cells  in  culture,  and  we  will 
determine  the  ran^  of  microenvironments  favored  by  breast  tumor  cells  from  multiple  (10-60)  individuals.  In 
addition,  we  will  determine  whether  the  drug  concentrations  and  exposure  times  used  in  previous  studies  were 
optimal,  and  we  will  determine  if  any  or  all  of  such  factors  could  have  contributed  to  fact  that  previously 
clinical  trials  have  reported  little  or  no  improvement  in  patient  survival  when  chemotherapy  was  selected  on  the 
basis  of  in  vitro  drug-sensitivity  tests. 
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Primary  Culture  of  Human  Breast  Tumor  Cells:  A  great  deal  of  progress  has  been  made  in  primary  culture 
of  malignant  human  breast  epithelial  cells  (BECs)  (Band  et  al.,  1990;  Ethier  et  al.,  1993;  Dairkee  et  al.,  1995; 
Dairkee  et  al.,  1997;  Li  et  al.,  1998).  Currently,  there  is  great  interest  in  the  unique  physiology  of  solid  tumors, 
and  it  is  likely  that  a  suboptimal,  nutritionally  depleted  environment  exists  in  breast  tumors  (Dairkee  et  al.,  1995; 
Tomida  and  Tsuruo,  1999;  Brown  and  Giaccia,  1998).  This  environment  is  believed  to  consist  of  regions  of  low 
oxygen  (hypoxia),  low  pH,  low  levels  of  glucose,  and  high  levels  of  metabolic  waste.  It  hiJs  been  demonstrated 
that  when  such  conditions  are  simulated  in  culture,  it  is  possible  to  isolate  relatively  pure  populations  of  primary 
breast  tumor  cells  (Dairkee  et  al.,  1995).  Nonmalignant  cells  are  unable  to  survive  these  initial,  hostile 
environmental  conditions.  Thus,  an  additional  benefit  is  that  nonmalignant  epithelium  with  its  higher 
proliferation  rate,  is  not  present  and  is  not  able  to  overgrow  the  tumor  cells  which  have  a  slower  proliferation 
rate.  A  common  misconception  is  that  malignant  BCEs  proliferate  more  rapidly  than  nonmalignant  BCEs.  It 
should  also  be  noted  that  drug  resistance  in  tumor  cells  may  depend  on  and  be  the  result  of  the  stress  of  a  hostile 
microenvironment  (Tomida  and  Tsuruo,  1999).  For  this  reason,  accurate  measurements  of  drug  resistance  may 
require  that  the  assays  be  performed  on  cells  in  the  proper  microenvironment.  It  is  interesting  to  note  that  some 
of  the  most  successful  in  vitro  chemosensitivity  tests  involve  the  culture  of  cells  in  tiny  capillary  tubes  that  are 
sealed  at  both  ends  and  incubated  for  14  days  (see  below).  The  resulting  microenvironment  is  likely  to  be  even 
more  hostile  than  those  generated  using  the  sandwiched  coverslip  technique  (Dairkee  et  al.,  1995). 

In  Vitro  Chemosensitivity  Assays:  The  idea  of  using  in  vitro  assays  to  select  chemotherapy  regimens 
tailored  to  the  tumor  of  the  individual  patient  is  not  new  (DeVita,  1997).  A  summary  of  clinical  correlations 
can  be  made  using  the  following  nomenclature:  TP  (true  positive,  patients  who  are  sensitive  in  vitro  and 
respond  to  therapy,  TN  (true  negative,  patients  who  are  resistant  in  vitro  and  do  not  respond  to  chemotherapy, 
FP  (false  positive,  patients  who  are  sensitive  in  vitro  but  resistant  clinically),  FN  (false  negative,  patients  who 
are  resistant  in  vitro  but  respond  clinically),  PPA  (positive  predictive  accuracy),  =  TP/(TP-FP),  percentage  of 
patients  with  sensitivity  in  the  test  who  respond,  NPA  (negative  predictive  accuracy)  =  TN/(TN  +  FN), 
percentage  of  patients  with  resistance  in  the  test  who  do  not  respond  to  therapy.  Correlations  of  in  vitro  test 
results  with  patient  response  show  an  overall  PPA  of  72%  and  a  NPA  of  90%.  This  corresponds  to  a  sensitivity 
(ability  to  detect  clinically  responsive  patients)  of  85%,  and  a  specificity  of  89%  (ability  to  detect  clinically 
unresponsive  patients).  It  should  be  noted  that  these  numbers  represent  an  average  of  7  different  assay  types  in 
studies  involving  4263  patients,  and  results  were  pooled  from  several  individual  studies  (DeVita,  1997).  These 
authors  have  suggested  that  these  assays  be  referred  to  as  drug-response  assays,  rather  than  chemosensitivity 
assays,  because  they  are  more  successful  at  predicting  patient  drug  resistance  than  predicting  patient  response. 

Clinical  Trials:  Three  prospective  clinical  trials  have  attempted  to  use  in  vitro  chemosensitivity  tests  to 
improve  patient  response  and  survival.  Both  have  failed  to  demonstrate  increased  survival  for  patients  who 
were  administered  chemotherapy  based  on  results  of  in  vitro  chemosensitivity  tests.  Two  different  in  vitro 
chemosensitivity  assays  were  used  in  these  studies,  the  HTCA  (human  tumor  cloning  assay)  and  the  DiSC 
(differential  staining  cytotoxicity,  or  Weisenthal  dye  exclusion)  assay.  The  in  vitro  cell  culture  and  propagation 
techniques  used  and  the  biological  endpoints  measured  differ  significantly  in  these  two  assays.  The  HTCA  calls 
for  the  culture  of  minced  tumor  tissue  in  very  small  capillary  tubes  that  are  sealed  at  both  ends  and  then 
incubated  for  14  days.  Before  this  mixture  of  malignant  and  nonmalignant  cells  (0.2-1. 0  x  10^  cells)is  sealed  in 
the  tubes,  the  cells  are  exposed  for  1  hour  to  standard  anticancer  agents  at  concentrations  corresponding  to  one- 
tenth  the  peak  plasma  concentration  seen  in  humans.Cells  are  then  suspended  in  0.3%  agar  and  are  sealed  in  100 
pL  capillary  tubes  for  14  days  at  37  ”C,and  7%  CO2.  After  14  days,  the  cells  are  extracted  from  the  tubes  and 
the  number  of  colpnies  are  manually  counted  under  a  microscope.  This  capillary  cloning  system  was  selected 
because  fewer  tumor  cells  are  initially  required  and  the  subsequent  outgrowth  of  colonies  with  significant 
numbers  of  cells  is  seen  to  be  improved.  The  DiSC  assay  is  a  nonclonogenic  assay,  where  tumor  cells  are 
cultured  in  liquid  medium  in  small,  polypropylene  tubes  for  4-6  days  order  to  amplify  the  number  of  cells 
available  for  testing.  Drug  exposures  times  range  from  1  hr  to  4  days,  depending  on  the  drug.  In  general,  drug 
concentrations  are  empirically  determined  and  are  higher  than  those  used  in  the  HTCA.  For  example,  cells  were 
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treated  with  doxorubicin  for  1  hr  at  0.04  i-ig/mL  in  the  HTCA  ,  but  for  1  hr  at  1 .2  |.ig/mL  in  the  DiSC  assay.  In 
the  Disc  assay,  cell  membrane  integrity  is  assessed  by  staining  dead  cells  in  suspension  with  Fast  Green. 
Acetaldehyde-fixed  duck  red  blood  cells  (DRBC)  are  added  to  the  culture  as  an  internal  standard,  and  then  the 
entire  mixture  is  cytocentrifuged  onto  a  microscope  slide,  with  the  result  that  living  cells  appear  clear,  whilel 
dead  cells  and  the  DRBC  are  stained  green.  Slides  are  then  counterstained  with  HE  (haematoxylin-eosin)! 
which  stains  the  living  cells.  Cells  are  then  identified  under  the  microscope  by  a  skilled  tedihnician  as  tumour  ori 
normal  who  counts  cells  to  determine  the  ratio  of  living  tumor  cells  over  DRBC. 

HYPOTHESIS/RATIONALE/PURPOSE 

Our  hypothesis  is  that  previous  clinical  studies  that  sought  to  administer  anticancer  drugs  based  on  in  vitro 
chemosensitivity  tests  failed  to  see  dramatic  increases  in  patient  response  or  survival  for  several  specific 
reasons:  (i)  the  microenvironment  favored  by  tumor  ceils  was  not  properly  simulated  in  vitro  (ii)  the 
physiological  responses  of  individual  cells  were  not  monitored  closely  throughout  the  assay,  and  (iii)  the  drug 
concentrations,  combinations,  and  exposure  times  examined  were  limited,  and  not  scientifically  optimized.  We 
further  hypothesize  that  Visual  Servoing  Optical  Microscopy  (VSOM)  can  rapidly  discover  and  exploit  key 
physiological  characteristics  which  distinguish  malignant  and  nonmalignant  human  breast  epithelial  cells 
(BECs).  In  this  way,  VSOM  will  help  to  solve  problems  (i-iii),  listed  above. 

OBJECTIVE/SPECIFIC  AIMS 

Demonstrate  and  verify  that  VSOM  can  rapidly  discover  and  optimize,  on  a  tumor-by-tumor  basis, 
microenvironmental  conditions  suitable  for  the  successful  in  vitro  propagation  and  chemosensitivity  testing  of 
primary  human  breast  tumor  cells.  Our  specific  aims  are  to: 

(1)  Rapidly  and  automatically  develop  in  vitro  microenvironments  that  favor  the  propagation  of 
primary  breast  tumor  cells 

(2)  Rapidly  develop  new  in  vitro  fluorescence  assays  that  predict  cell  behavior  on  the  basis  of  early 
physiological  responses  to  applied  stress 

(3)  Improve  VSOM  technology  in  order  to  achieve  our  objectives  and  specific  aims. 

PRELIMINARY  DATA 

Visual  Servoine  Optical  Microscopy:  A  VSOM  fluorescence  assay  consists  of  (i)  real-time  observations  of 
individual  cell  responses  to  a  series  of  environmental  perturbations  or  stimuli  (ii)  real-time,  automatic,  and 
intelligent  adjustment  of  these  stimuli  and  environmental  perturbations  in  order  to  achieve  specified  goals. 
Currently,  environmental  perturbations  and  stimuli  are  applied  by  means  of  computer-controlled  perfusion 
pumps  (Figure  1).  Individual  cell  responses  are  monitored  using  a  variety  of  fluorescence  probes.  An  example 
of  a  specified  goal  might  be  the  identification  of  a  particular  subpopulation  of  cells  based  on  an  observed  stress 
response  as  the  pH  of  the  extracellular  medium  is  decreased.  An  intelligent  response  of  the  sytem  would  be  to 
lower  the  pH  of  the  medium  until  the  cells  of  interest  had  been  identified,  then  return  the  cells  to  a  higher  pH 
before  they  are  irreversibly  damaged.  The  cells  of  interest  would  then  be  identified  on  a  cell-by-cell  basis,  and 
the  system  could  then  proceed  with  additional  tests  that  attempt  to  optimize  or  exploit  this  differential  response 
to  a  low  pH  environment. 

'( 

An  ideal  VSOM  fluorescence  assay  for  cell  identification  is  performed  rapidly  on  a  complicated  mixture  of 
cells,  the  fluorescent  probe  is  nontoxic  to  the  cells,  cells  are  identified  based  on  characteristic  physiological 
responses  to  applied  stimuli  or  perturbations,  and  the  stimuli  or  perturbations  do  not  have  an  irreversible  effect 
on  the  cells.  In  our  opinion,  the  most  desirable  predictive  assays  would  be  dynamic  and  adaptive  VSOM 
fluorescence  assays  involving  the  observation  of  a  series  early  physiological  responses  to  intelligently  chosen. 
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computer-controlled  stimuli.  This  approach  becomes  more  powerful  when  such  stimuli  and  perturbations  are 
reversible  and  do  not  permanently  commit  the  cell  to  a  given  biological  path.  The  ability  to  repeatedly  apply 
relatively  innocuous  stimuli  and  perturbations  to  living  cells  will  allow  us  to  repeatedly  probe  the  cell  for 
physiological  characteristics  that  can  be  exploited  by  anticancer  drugs  or  combinations  of  anticancer  drugs.  Our 
initial  goal  is  to  develop  VSOM  fluorescence  assays  that  are  highly  predictive  of  the  cell's  future  behavior  or 
ultimate  fate;  for  example,  whether  the  cell  has  been  sufficiently  stimulated  to  proliferate,  dfr  whether  it  has  been 
stressed  to  the  point  where  apoptosis  is  inevitable.  This  capability  will  help  us  achieve  our  objective  of  making 
correlations  between  the  early  physiological  responses  observed  in  cells  and  the  future  biological  state  or 
ultimate  biological  fate  of  the  cells,  whether  this  be  proliferation,  apoptosis,  or  arrest  in  some  stage  of  the  cell 
cycle. 

Our  current  VSOM  is  shown  in  Figures  1  and  2.  Currently  four  syringe  perfusion  pumps  are  under  computer 
control  (Fig.  lA),  and  we  have  modified  a  temperature-controlled  microincubation  chamber  so  that  it  mounts 
firmly  to  our  computer-controlled  xy  scanning  stage  (Fig.  2A).  A  vacuum  powered  aspirator  removes  excess 
liquid  and  a  temperature  probe  provides  feeback  for  temperature  control  (Fig.  2B).  More  details  on  this 
hardware  can  be  found  in  the  Facilities  Description  pages.  As  seen  schematically  in  Figure  3,  the  system 
currently  references  its  position  relative  to  an  extremely  fine  grid  ("+'')  that  is  gently  etched  into  the  sterile, 
plastic  culture  dish  using  a  glazier's  microfinish  wheel.  The  dish  can  then  be  removed  and  placed  back  into  the 
incubator.  When  it  is  time  to  reimage  the  cells,  the  system  is  calibrated  by  making  reference  to  the  "+"  on  the 
dish,  and  the  (0,0)  point  of  the  coordinate  is  taken  to  be  the  center  of  the  "+".  The  user  can  easily  see  the 
etching  under  transmitted  light  for  initial  positioning. 

A  simple  VSOM  experiment,  currently  in  use  for  determining  the  degree  of  multidrug  resistance  in 
individual  MCF-7  cells,  is  shown  schematically  in  Figure  4.  As  seen  in  Figure  4A,  the  nuclei  of  living  cells  can 
be  prelabeled  using  the  membrane  permeable  fluorescence  dye  Hoechst  33342  (H42).  As  seen  in  Figure  4B, 
after  Lysotracker-Red  (LT-R)  fluorescence  dye  (a  lysosome-specific  organelle  stain  for  living  cells)  has  been 
perfused  into  the  chamber,  individual  cell  lysosomes  become  labeled.  The  mean  fluorescence  intensity  of  each 
cells  in  one  field  of  view  is  monitored  using  multiple  fluorescence  channels,  corresponding  to  multiple 
fluorescence  excitation  and  emission  wavelengths.  These  quantitative  measurements  for  an  actual  experiment 
are  shown  in  Figure  5  for  MCF-7.  cells.  The  error  bars  indicate  ±  standard  deviation  for  325  cells.  The  blue 
symbols  represent  a  steady  H42  signal  in  all  the  cells  as  a  function  of  time.  However,  the  red  signal 
representing  LT-R  uptake  is  seen  to  Increase  as  soon  as  LT-R  perfusion  begins.  At  approximately  6000  s,  the 
LT-R  pump  is  turned  off,  the  Calcein-AM  (CAM)  perfusion  is  begun,  and  green  fluorescence  signal  begins  to 
accumulate  in  the  cells,  while  LT-R  begins  to  be  rinsed  out  of  the  cell.  Calcein-AM,  like  LT-R  is  a  cell  viability 
stain  that  is  only  retained  in  cells  with  intact  membranes.  Multidrug-resistant,  or  MDR  cells,  however,  actively 
pump  CAM  out  of  the  cell.  CAM  signal  appears  thoughout  the  cytoplasm  as  depicted  in  Figure  4C.  When  the 
LT-R  signal  drops  below  the  CAM  signal,  the  CAM  pump  is  turned  off,  and  the  LT-R  pump  is  turned  back  on 
until  the  LT-R  intensity  reaches  its  previous  maximum. 

It  should  be  noted  that  an  entire  digital  image  is  taken  for  each  fluorescence  channel  (red,  green,  and  blue) 
and  an  image  is  also  taken  using  transmitted  light.  At  each  time  point,  a  full-color  image  can  be  constructed  as 
shown  in  Figure  7.  In  this  experiment  long  exposures  at  345nm  light  were  made  repetively  until  the  cell 
membrane  blebbed,  and  the  cell  lysed,  resulting  in  a  loss  of  the  red  LT-R  stain,  and  the  green  CAM  stain.  This 
time-dependent  process,  and  a  large  number  of  lysed  cells  may  be  seen  by  comparing  Figures  4A,C  to  Figures 
B,  D. 

We  can  also  monitor  cells  in  multiwell  plates  using  either  the  microscope  (Fig  6A)  or  a  fluorescence  plate 
reader  (Figure  6B).  The  process  of  loading  such  plates  with  compounds  such  as  doxorubicin  (DOX)  is  shown 
in  Figure  6C.  We.^plan  to  control  the  pH  and  p02  during  these  plate  exposures  by  putting  the  plates  into  special 
gas  chambers  which  are  then  placed  in  the  incubator  (Figure  6D,E).  Gas  mixtures  of  known  C02  and  02 
composition  can  then  be  perfused  into  these  gas  chambers  to  control  pH  and  p02  during  long  term  incubations. 
An  image  of  MCF-7  cells  in  one  well  of  a  plate  is  shown  in  Figure  6F.  These  cells  are  stained  with  a  live/dead 
stain  (CAM,  and  ethidium  homodimer,  or  EH).  Thus,  green  cells  are  alive  and  red  cells  are  dead.  We  have  also 
developed  fluorescence  staining  procedures  for  detecting  mRNA  expression  in  living  cells.  LDS-751 
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(Molecular  Probes,  Eugene  OR)  stains  both  QNA  and  RNA.  However,  a  large  spectral  blue  shift  takes  place 
when  LDS-751  moves  from  being  bound  to  DNA  (infrared  fluorescence  emission)  to  being  bound  to  mRNA. 
Thus,  as  mRNA  levels  increase  in  the  cells  in  response  to  an  agent  that  increases  intracellular  calcium 
(thapsigargin,  or  TG)  the  resulting  red  fluorescence  becomes  visible  in  our  red  channel  (Figures  8A,B).  It 
should  be  noted  that  the  cells  can  also  be  labeled  with  FURA-2-PE3,  a  calcium  sensitive  fluorescence  dye. 
Thus,  one  can  observe  increases  in  intracellular  calcium  at  the  same  time  one  is  monitoring  the  expression  of 
mRNA  in  another  channel. 

Intracellular  calcium  is  monitored  by  performing  image  ratioing.  One  takes  a  digital  image  at  345nm  I 
excitation,  and  divides  this  image,  on  a  pixel-by-pixel  basis,  by  a  second  image  that  was  acquired  at  385nm  ! 
excitation  (Negulescu  et  al.,  1994).  It  has  been  demonstrated  in  T  lymphocytes  that  one  can  stimulate  cells  with 
TG,  and  then  perform  perfusions  with  high  or  low  calcium  buffer.  In  this  way,  one  can  control  the  intracellular 
levels  of  calcium.  If  the  levels  of  intracellular  calcium  are  elevated  for  a  sufficiently  long  time,  subsequent  gene 
expression  can  be  observed  in  response  to  this  perceived  intracellular  signal  (Negulescu  et  al.,  1994).  Using 
the  calcium  image  rationing  technique,  it  has  also  been  demonstrated  that  the  proliferation  rate  of  primary 
cultures  of  human  mammary  epithelial  cells  (HMECs)  can  be  increase  as  intracellular  levels  of  calcium  are 
increased(Tannheimer  et  al.,  1997).  In  this  study,  polycyclic  aromatic  hydrocarbons  (PAHs)  were  used  to 
elevate  intracellular  levels  of  calcium  and  thus  generate  an  increase  in  the  proliferation  rate  of  the  cells.  Image 
ratioing  can  also  be  used  to  detect  BrdU  incorporation.  As  seen  in  Figure  9,  it  is  possible  to  quantify  the  degree 
of  BrdU  incorporation  in  living  cells.  Using  this  technique,  we  have  verified  the  effects  of  EGF  stimulation  on 
184B5  cells,  an  immortalized  HMEC  cell  line  (Stampfer  et  al.,  1993).  Using  the  same  cells  (after  fixation  with 
acidic  ethanol)  shown  in  Figure  9,  we  also  performed  indirect  immunofluorescence  detection  of  BrdU 
incorporation  using  an  antiBrdU  antibody.  Similar  results  were  obtained  (data  not  shown). 

Image  Processing 


The  process  of  visual  servoing  is  highly  dependent  on  our  ability  to  detect  and  outline  individual  cells  in 
digital  images.  This  is  referred  to  as  "segmenting"  the  image.  We  currently  use  two  different  segmentation 
algorithms  for  stained  nuclei.  We. are  using  these  algorithms  for  our  live  cell  studies,  and  for  our  studies  of 
immunostained  tissue  sections.  We  will  show  examples  of  both  types  of  images.  One  segmentation  approach  is 
based  on  a  variation  of  Hough  transform  and  is  suitable  for  a  large  class  of  cells  as  long  as  they  conform  to  a 
specifically  parameterized  geometric  model.  The  second  approach  focuses  on  extraction  of  partial  information 
corresponding  to  step  and  roof  edges  and  then  groups  them  through  geometric  reasoning  (Cong  and  Parvin, 
1999  June;  Cong  and  Parvin,  2000).  The  intent  is  to  construct  a  partitioning  that  is  globally  consistent.  A  list  of 
step  edges  is  constructed  by  thresholding  raw  images,  recovering  corresponding  boundaries,  and  localizing 
concave  corners  from  their  polygonal  approximation.  These  comers  provide  possible  cues  to  the  position 
where  two  adjacent  nuclei  may  touch  each  other.  Thresholding  separates  big  clumps  consisting  of  several 
nuclei  squeezed  together,  but  the  boundaries  between  adjacent  nuclei  cannot  be  detected  because  they  have 
higher  intensities  than  background.  The  partial  boundaries  between  adjacent  nuclei  can  be  recovered;  however, 
by  extracting  crease  segments  (Monga  et  al.,  1992;  Thirion,  1996;  Thirion  and  Gourdon,  1996).  These  crease 
segments  correspond  to  trough  edges  (positive  curvature  maxima),  but  false  creases  may  also  be  extracted  in 
the  process.  Nevertheless,  dealing  with  noise  and  erroneous  segments  is  a  higher-level  process  that  is  handled 
during  the  grouping  process.  Our  system  generates  a  number  of  hypotheses  for  possible  grouping  of  the 
boundary  segments.  A  unique  feature  of  this  system  is  in  hyperquadric  representation  of  each  hypothesis  and 
use  of  this  representation  for  global  consistency.  The  main  advantage  of  such  a  parameterized  representation 
(as  opposed  to  polygonal  representation)  is  compactness  and  better  stability  in  shape  description  from  partial 
information.  In  this  fashion,  each  step-edge  boundary  segment  belongs  to  one  and  only  one  nucleus  while  each 
trough-edge  boundary  segment  is  shared  by  two  and  only  two  nuclei.  These  initial  hypotheses  and  their 
localized  inter-relationship  provide  the  basis  for  search  in  the  grouping  step.  This  is  expressed  in  terms  of  an 
adequate  cost  function  and  minimized  through  dynamic  programming.  The  final  result  of  this  computational 
step  is  then  shown  to  an  operator  for  verification  and  elimination  of  false  alarms.  An  example  is  shown  in 
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Figure  10,  where  comparison  to  manual  segmentation  is  shown.  A  major  limitation  of  this  approach  is  its 
computational  complexity.  Grouping  of  local  features  is  a  complex  process,  and  simpler  techniques  are  highly  ' 
desirable. 

1 

Recently,  we  have  developed  three  new  techniques.  One  is  based  on  a  regularized  form,  the  second  on  a  > 
vector  representation  of  the  underlying  image  followed  by  feature  extraction  in  vector  spafce,  and  the  third  on  a 
geometric  and  texture  model  of  a  cell  and  its  interaction  with  neighborhoring  cells.  The  new  regularization  form 
has  its  root  in  visualization  of  3D  objects;  however,  we  have  also  found  it  to  be  applicable  to  segmentation  of 
scientific  images.  The  vector  representation  of  images  has  its  root  in  analysis  of  spatio-temporal  satellite  images 
(Yang  and  Parvin,  2000a;  Yang  and  Parvin,  2000b).  The  detection  result  using  the  regularized  form  is  shown  in 
Figure  1  lA  and  B.  The  third  approach  has  been  developed  for  measuring  mRNA  expression.  Due  to  space 
limitations  we  can  only  show  our  preliminary  results.  These  preliminary  results  are  shown  in  Figures  1 1  and 
12. 

PROPOSED  RESEARCH  AND  METHODS 

The  cells  lines  available  for  our  use  have  been  previously  described  (Dairkee  et  al.,  1995;  Dairkee  et  al., 
1997;  Li  et  al.,  1998).  To  summarize,  the  number  and  types  of  specimens  available  are;  organoids  for  epithelial 
propagation  that  were  derived  from  reduction  mammoplasties  (82  cases);  epithelial  cultures  from  primary  breast 
carcinoma  at  passage  2  or  greater  (73  cases);  matched  epithelium  from  non-malignant  peripheral  tissue  (23 
cases).  These  human  specimens  are  not  the  same  ones  used  in  the  previous  prospective  trials,  nor  are  they  tied 
to  a  patient  database.  Our  approach  is  to  build  on  the  data  and  findings  of  the  previous  prospective  trials.  This 
approach  allows  us  to  focus  on  breast  cancer,  rather  that  lung  cancer  (Wilbur  et  al.,  1992;  Gazdar  et  al.,  1990), 
or  "cancer"  in  general,  as  was  the  approach  of  the  HTCA  studies(Von  Hoff  et  al.,  1990).  For  example,  in  the 
HTCA  study  only  5  breast  cancer  specimens  were  used  and  this  would  not  be  a  sufficient  number  of  individuals 
for  our  proposed  studies.  For  an  intial  estimate  of  how  many  individual  specimens  we  will  need  to  examine,  we 
will  perform  studies  on  the  tumor  cells  derived  from  10  individuals  We  will  compare  differences  in  the 
proliferation  rates  of  tumor  cells  from  different  individuals  and  determine  how  these  rates  can  be  increased,  on  a 
per  patient  basis,  as  the  microenvironment  is  altered.  It  is  our  hypothesis  that  this  important  experimental 
parameter,  the  microenvironment  in  which  the  drug  tests  are  performed  was  not  controlled  closely  enough  in  the 
previous  studies.  In  order  to  determine  how  sensitive  such  tumor  cell  proliferation  and  drug  response  tests  are 
to  the  preferred,  or  optimum,  microenvironment,  (which  we  believe  will  vary  from  individual  to  individual)  we 
will  use  tumor  cells  derived  from  at  least  10,  and  at  most,  70  patients.  In  addition,  we  will  perform  parallel 
studies  on  at  least  10  specimens  of  nonmalignant  BCEs,  derived  from  reduction  mammoplasties.  We  expect  to 
see  much  less  variation  in  the  preferred  microenvironment  when  we  examine  these  nonmalignant  BCEs. 

There  are  additional  advantages  to  using  well  characterized  primary  breast  tumor  cells.  For  example,  we  will 
already  have  information  concerning  the  erb-B2,  p53,  and  bcl-2  status  of  some  cases  (Dairkee  et  al.,  1997) .  It  is 
likely  this  will  help  us  interpret  observed  differences  in  specimens;  for  example,  the  relative  resistance  or 
sensitivity  of  different  specimens  to  apoptosis. 

(NOTE:  Due  to  space  limitations,  we  will  not  repeat  experimental  details  or  the  experimental  timeline  that 
already  appears  in  the  Statement  of  Work). 

Experimental  Approach;  Our  approach  will  be  to  monitor  intracellular  calcium  levels,  intracellular  pH, 
mitochondrial  membrane  potentials,  mRNA  levels,  and  BrdU  incorporation  in  individual  living  cells  as  various 
growth  stimuli  and/or  environmental  stresses  are  applied.  Early  physiological  responses  to  perturbations  of  the 
microenvironment/  (pH,  p02,  [Ca^"]ex  ,anticancer  drugs)  will  be  stored  in  our  bioinformatics  database.  The 
ultimate  effect  of  these  perturbations  will  be  assessed  by  using  VSOM  to  return  to  the  same  field  of  cells  after 
longer-term  incubations.  In  addition,  responses  that  appear  to  predict  an  increase  in  proliferation  or  apoptosis 
will  be  verified  in  multiwell  plate  fluorescence  assays  during  longer-term  incubations.  Proliferation,  cell 
viability,  and  apoptosis  will  be  monitored  using  VSOM  and  multiwell  plate  fluorescence  assays.  This  approach 
will  allow  us  to;  (1)  develop  conditions  that  are  favorable  or  selective  for  the  propagation  of  primary  breast 
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tumor  cells  in  culture,  (2)  develop  an  array  of  predictive  assays  based  on  observations  of  dynamic,  early 
physiological  events  (3)  document  tumor  cell  mechanisms  of  resisting  effects  of  anticancer  drugs,  (4)  use  this 
information  to  pick  the  proper  combination  of  antitumor  drugs 

Anticancer  Drugs:  The  drugs  we  will  use  are  those  that  were  used  in  all  three  of  the  previous  clinical  trials. 
Concentrations  are  given  in  pg/mL,  followed  by  exposure  time  where  C  indicates  continuous  exposure  for  th'' 
duration  of  the  test.  Our  initial  experiments  will  use  this  range  of  concentrations  and  exposifre  times. 


DRUG 

Disc 

HTCA 

cisplatin 

1.65,  C 

0.2,  1  hr 

doxorubicin 

1.2,  Ih 

0.04,  1  hr 

etoposide 

41.7,  C 

3.0,  Ihr 

5-fluorouracil 

20.0,  C 

6.0,  Ihr 

mitomycin  C 

0.30,  C 

0.1, 1  hr 

vinblastine 

0.55,  C 

0.05, 1  hr 

vincristine 

0.25,  C 

0.01,  Ihr 

Cell  stresses  other  than  anticancer  drugs:  Agents  that  increase  [Ca'^'jin  with  used  alone  or  in  conjunction  with 
high  and  low  calcium  buffer  exchanges.  These  agents  include  TG,  calcium  ionophores,  and  PMA  (phorbol  12- 
myristate  13-acetate),  as  previously  described  (Negulescu  et  al.,  1994).  Low  pH  will  be  induced  during  VSOM 
experiments  by  constant  perfusion  of  buffers  of  known  pH.  Low  p02  will  induced  during  VSOM  experiments 
by  (i)  perfusing  solutions  that  were  previously  equilibrated  to  a  known  pOi  by  bubbling  gas  of  known 
composition  through  the  solution.  In  addition,  gas  of  the  proper  composition  will  be  layered  over  the  cell 
chamber  during  the  experiment.  Our  cell  chamber  is  designed  for  this  purpose. 

Other  agents  to  be  investigated:  EGF,  TGF-P 

Early  responses  to  be  monitored:  intracellular  calcium,  intracellular  pH,  mitochondrial  membrane  potential, 
total  mRNA  levels,  levels  of  reactive  oxygen  species,  and  changes  in  morphology  via  transmitted  light 
observations.  Intracellular  calcium  will  be  monitored  using  calcium  ratio  imaging  in  cells  that  have  been  pre- 
loaded  with  FURA-2-PE3.  Intracellular  pH  will  be  monitored  using  cells  labeled  with  BCECF,  and  image 
ratioing  will  be  used  here,  also.  Mitochondrial  membrane  potential  will  be  monitored  using  TMRE,  as 
previously  described  (Farkas  et  al.,  1989).  Total  mRNA  will  be  monitored  using  LDS-751,  as  described  in  the 
preliminary  data  section.  Levels  of  reactive  oxygen  species  will  be  monitored  using  a  variety  of  fluorescence 
probes  that  are  sold  for  this  purpose  by  Molecular  Probes  (Eugene.  ORL  We  are  especially  interested  in  the 
generation  of  reactive  axygen  species  generated  by  TGF-p. 

Biological  Endpoints:  proliferation,  apoptosis,  necrosis.  Proliferation  will  be  monitored  via  BrdU 
incorporation  and  nuclear  image  ratioing  as  described  in  the  preliminary  data  section.  A  wide  variety  of 
fluorescence  probes  are  available  from  Molecular  Probes  (Eugene,  OR).  The  live/dead  assay  described  in  the 
Preliminary  Data  section  will  also  be  used. 

Multiwell  Plate  experiments:  For  long  term  incubations  of  amplified  cells  we  will  use  multiwell  plate  assays. 
We  will  use  a  fluorescence  plate  reader,  and  also  image  cells  in  the  individual  wells  as  demonstrated  in  the 
Preliminary  Data  section.  Fluorescence  assays  kits  for  cell  proliferation,  etc.  are  commercially  available.  In 
cases  where  p02  or  pH  must  be  held  constant,  multiwell  plates  will  be  placed  in  gas  chambers  as  described  in 
the  Preliminary  Data  section.  Gas  of  known  O2  and  CO2  composition  will  be  continously  perfused  into  these 
chambers  in  order  to  control  p02  and/or  pH. 

Cell  Sedimentation  Manifolds:  We  have  obtained  a  set  of  homemade  cell  sedimentation  chambers,  provided  to 
us  by  Dr.  Mike  Bqrens  of  Creative  Scientific  Methods,  Inc.  (www.cre8ive-sci.com).  The  procedure  for  using 
these  chambers  to  deposit  very  small  numbers  of  cells  onto  small,  well-defined  regions  of  different  surfaces  is 
described  on  this  company's  website. 

Fluorescence  Calibrations:  Multispeck  beads  will  be  used  as  fluorescence  intensity  standards  (Molecular 
Probes,  Eugene,  OR).  Defined  standard  calcium  solutions  are  also  provided  by  this  company  for  calcium  ratio 
imaging  calibrations. 
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BEC 

[Ca"-]ex 

[Ca"-]in 

CAM 

Disc 

DOX 

DRBC 

DS 

EH 

FN 

FNA 

FP 

FURA-2-PE3 

H42 

LT-R 

HMEC 

HTCA 

MCF-7 

MDR 

NPA 

PMA 

PPA 

TRME 

TN 

TP 

TG 

VS 

VSOM 


breast  epithelial  cell 
extracellular  calcium  concentration 
intracellular  calcium  concentration 
calcein-AM 

differential  staining  cytotoxicity 

doxorubicin 

duck  red  blood  cells 

drug  sensitive 

ethidium  homodimer 

false  negative 

fine-needle  aspirate 

false  positive 

a  calcium  sensitive  fluorescence  dye 

Hoechst  33342 

LysoTracker  Red 

human  mammary  epithelial  cell 

human  tumor  cloning  assay 

a  human  breast  cancer  cell  line 

multidrug-resistant 

negative  predictive  accuracy 

phorbol  12-myristate  13-acetate 

positive  predictive  accuracy 

tetramethylrhodamine  ethyl  ester 

true  negative 

true  positive 

thapsigargin 

visual  servoing 

visual  servoing  optical  microscopy 
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Cat  # 

Description 

Price 

AH  73-0235 

HSE-Harvard  Data  Acquisition  Hardware,  W98 

$1834.25 

AH-73-0045 

PLUGSYS  Basic  System  Case,  Type  603 

$1890.00 

AH-73-0212 

EMM  Amplifier  Module 

$1420.00 

AH-73-0215 

pHMM  Amplifier  Module 

$1420.00 

AH-73-0210 

OPPM  Oxygen  Partial  Pressure  Module 

$2060.00 

AH-73-0208 

Flow-Through  02  Chemosensor 

$1515.00 

AH-73-2712 

Reference  Electrode 

$  955.00 

AH-73-0213 

pH  Flow-Through  Sensor 

$1384.60 

AH-73-2711 

Flow-Through  Biosensor  for  glucose,  L-Lactate,  etc. 

$1515.00 

AH-73-0642 

Flow-Through  Sensor  for  Calcium  Ion 

$1696.83 

Callahan,  Daniel  E. 
Subtotal 


$15690.68 


Equipment  budget  justification: 

We  have  requested  $15,700  for  biosensors  and  the  associated  electronics  necessary  for  on-line  monitoring  of  important 
microenvironmental  parameters  during  VSOM  experiments.  The  sensors  we  have  chosen  are  ZABS  flow-through  sensors 
that  can  be  connected  in  series  to  our  perfusion  lines.  Thus,  we  will  be  able  to  continously  monitor  and  log  several 
important  properties  of  the  medium  Just  before  it  reaches  the  ceils.  There  is  not  enough  room  to  place  all  these  sensors  in 
the  actual  cell  perfusion  chamber,  due  to  the  presence  of  the  large  transmitted  light  condenser,  the  temperature  probe,  and 
the  vacuum  aspirator.  Therefore,  we  will  monitor  pH,  L-lactate  (or  glucose),  extracellular  calcium,  and  pOa  of  the 
medium  as  it  is  continuously  perfused  into  our  microincubation  chamber.  The  electronics  package  (PLUGSYS)  we  have 
selected  is  modular  in  design  and  the  basic  system  case  has  sufficient  slots  to  hold  the  various  amplifier  modules 
necessary  for  the  measurement  of  pH,  p02  ,  calcium,  and  other  biological  molecules  such  as  glucose  or  L-lactate.  The 
PLUGSYS  system  case  is  also  compatible  with  the  data  acquisition  hardware  we  have  chosen,  which  includesa  PCI  A/D 
converter  board  for  a  PC.  This  equipment  will  provide  important,  real-time  data  for  VSOM  system  control,  and  will  allow 
us  to  continously  monitor  and  log  microenvironmental  parameters  that  are  important  for  the  execution  and  interpretation 
of  our  proposed  experiments. 

Supply  budget  justification: 

A  significant  portion  of  the  requested  supply  money  is  for  the  purchase  of  fluorescence  dyes  from  Molecular  Probes 
(Eugene,  OR).  For  example,  a  great  deal  of  the  fluorescent  dye  calcein-AM  is  consumed  during  on-line  VSOM 
experiments  where  100  mL  of  a  1.0  pM  solution  is  perfused  through  the  cell  chamber  during  a  single  experiment.  In 
addition,  this  compound  must  be  repeatedly  added  to  all  the  wells  of  multiwell  plates  in  many  of  our  proposed 
experiments.  The  cost  of  this  compound  is  $142  for  1  mg  (Catalog  #C-3099,  Img,  MW  =994g/mole).  In  addition,  a 
significant  amount  of  sterile  culture  work  will  be  performed  on-site  at  LBNL,  and  the  sterile  serological  pipettes,  tissue 
culture  flasks,  media,  serum,  multiwell  plates,  and  tissue  culture  dishes  are  expensive  consumables.  In  addition,  we 
require  sterile  syringes  and  tubes  of  various  sizes,  special  pipette  tips  and  sterile  reservoirs  for  our  multichannel  pipettor, 
0.2  pm  syringe  filters  for  the  preparation  of  sterile  solutions,  sterile  chambered  coverslips  for  high-resolution,  high 
magnification  oil-immersion  objective  studies  of  cells,  and  replacement  arc  lamps  and  excitation  filters  for  the 
fluorescence  microscope. 
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Personell  and  subcontract  budget  justification: 


Callahan,  Daniel  E. 


This  is  a  multi-institution,  multi-disciplinary  proposal.  Dr.  Callahan's  effort  is  required  at  the  50%  level.  He  will  . 
coordinate,  schedule,  and  perform  the  VSOM  experiments  and  the  multiwell  plate  fluorescence  assays  on-site  at  LBNL 
using  living,  human  BECs  cells  provided  by  Dr.  Shanaz  H.  Dairkee.  Dr.  Callahan  will  test,  modify  and  refine  VSOM 
fluorescence  assays  and  protocols  and  integrate  the  various  hardware  components  into  a  functioning  system.  Dr.  Callahan  j 
will  test  and  help  refine  the  VSOM  image  analysis,  data  reduction,  and  system  control  software  provided  by  Dr.  Bahram  j 
Parvin.  These  evaluations  will  be  conducted  both  on-line  during  VSOM  experiments,  and  off-line  after  the  experiments  1 
are  completed.  Dr.  Callahan  will  make  the  image  data  and  analysis  results  available  to  all  members  of  the  group.  Dr. 
Callahan  will  manage  and  backup  the  data  using  magnetic  tape,  and  will  keep  records  of  experiments  performed, 
summaries  of  results,  and  records  that  track  data  sets  stored  on-line  and  off-line.  Dr.  Callahan  will  also  coordinate  the 
overall  group  effort  regarding  publications,  and  presentations  and  will  publish  biological  results  in  the  relevant  biological 
journals.  Dr.  Parvin's  effort  is  required  at  the  10%  level.  Dr.  Parvin  will  provide  software  necessary  to  improve  the 
exisiting  VSOM  system  software,  and  will  publish  computer  science  results  in  the  relevant  computer  science  journals.' 
Dr.  Parvin  will  eventually  control  the  VSOM  system  remotely,  and  will  store  digital  images,  singletCell  responses,  and 
relevant  experimental  parameters  directly  into  a  modern,  object-oriented  bioinformatics  database.  Dr.  Parvin  will  make 
this  database  available  to  other  members  of  the  group  and  this  will  allow  them  to  examine  and  visualize  the 
multidimensional  data  in  ways  that  will  generate  novel  experimental  insights.  Dr.  Shanaz  Dairkee  will  provide  a  variety 
of  well-characterized  human  BECs.  both  malignant  and  nonmalignant.  for  VSOM  and  multiwell  plate  experiments.  Her 
effort  is  required  at  the  10%  level.  In  addition  to  cells,  she  will  provide  critical  cancer  cell  biology  and  cell  culture 
expertise  to  the  group  and  will  help  design  and  interpret  experiments  on  a  regular  basis.  She  will  also  monitor  the  group's 
progress  from  the  perspective  of  a  cancer  biologist,  and  will  help  set  experimental  priorities  so  that  important  biological 
results  are  rapidly  and  efficiently  generated.  (Modify  text  for  Dr.  Dairkee) 


Callahan,  Parvin  Page  91 


Page  16 


FACILITIES 
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Laboratory  Space:  Dr.  Callahan’s  shared  lab  space  consists  of  600  sq.  ft.  containing  4  work  benches,, 
refrigerators  and  freezers,  2  sinks,  laminar  flow  hood,  3  incubators,  cell  culture  hood,  chemical  storage,* 
networked  Pentium  computers  and  printers.  Major  equipment  includes  a  computer  controlled  fluorescence 
multiwell  plate  reader  (Cytofluar  4000)  and  Cytocalc  sofware,  automatic  microplate/*  strip  washer,  PCR 
thermocycler  (Crocodile  II),  gel  electrophoresis  apparatus,  Nikon  inverted  phase  microscope.  Coulter  counter, 
balances,  water  baths,  centrifuges,  table  top  and  ependorf  centrifuges.  In  addition.  Dr.  Callahan  has  a  complete 
wet  laboratoo'  and  cell  culture  facility  that  is  dedicated  to  VSOM  studies.  This  facility  is  immediately  adjacent 
to  the  Life  Sciences  Microscope  Resource  (LSMR)..  In  addition,  he  is  part  of  the  Barcellos-Hoff  Group  and 
has  access  to  all  her  labs  and  equipment 

Dr.  Callahan  has  access  to  several  computers.  He  shares  a  Pentium  111  PC  and  a  Sparstation  Ultra  1  with 
members  of  the  Barcellos-Hoff  research  group,  and  also  has  access  to  the  shared  SUN  Sparcstations  in  the  Life 
Sciences  Microscope  Resource.  All  the  computers  are  networked  and  have  access  to  the  centrally  located 
(Building  50)  LBNL  sotware  farm  and  other  computer  resources  located  in  Dr.  Parvin’s  building  at  the  LBNL 
Supercomputing  Center  (NERSC). 

Digital  Imaging  Fluorescence  Microscopy:  Studies  will  be  performed  at  the  Life  Sciences  Microscope 
Resource  (LSMR,  in  room  174,  Building  74)  using  our  Visual  Servoing  Optical  Microscope  (VSOM).  This 
Zeiss  Axiovert  135  H/DIC,  TV  inverted  microscope  is  used  for  transmitted  light  (phase  and  DlC)  and 
multicolor  fluorescence  microscopy.  It  is  equipped  with  a  computer  controlled  xy  scanning  stage,  z-axis 
stepping  motor,  and  six  position  filter  wheel.  A  12-bit  Xillix  CCD  camera  (Xillix  Technologies,  Vancouver, 
BC)  containing  a  Kodak  KAF-1400  CCD  chip  (1317x1035  pixels,  7x7  micron  pixel  size)  will  be  used  for  these 
studies.  This  camera  has  a  readout  rate  of  8  MHz  (approximately  four  full  size  images  per  sec).  Images  from 
the  cameras  are  readout  directly  into  a  Sparcstation  Ultra  1,  a  multitasking  UNIX  workstation  that  can  store  or 
transfer  images  across  the  network  during  the  course  of  the  experiment.  A  100  BaseT  network  line  links  the 
LSMR  computers  and  microscope  to  the  computers  of  the  Imaging  Technology  and  Distributed  Computing 
Group  (Dr.  B.  Parvin,  Building  50).  A  Mass  Storage  System  for  digital  images  is  also  available  at  the  LBNL 
Supercomputer  Center  (NERSC,  Building  50).  The  VSOM  system  is  built  around  an  inverted  microscope  and  a 
vibration  isolation  table  is  installed.  This  inverted  microscope  has  a  base  port  so  that  cameras  mounted  below 
obtain  better  fluorescence  signal.  In  addition,  the  inverted  design  allows  perfusion  of  living  cells,  with  cells 
located  on  the  bottom  of  a  microperfusion  cell  chamber.  With  this  design  it  is  possible  to  switch  from 
fluorescence  to  transmitted  light  detection  (phase  or  DIC)  and  verify  cell  locations,  morphology,  etc.  This  is 
achieved  with  a  computer  controlled  transmitted  light  shutter.  A  computer  controlled  epifluorescence  shutter  is 
also  available  in  the  filter  wheel.  Operation  of  the  system  (control  of  shutters,  filter  wheel,  xy  stage,  z-axis 
motor,  images  acquisition  )  can  be  automated  using  the  software  SCIL  Image  (TNO  Institute  of  Applied 
Physics,  Delft,  The  Netherlands),  or  using  the  VSOM  software  of  Dr.  Parvin..  Two  computer  controlled,  dual 
syringe  pumps  (Harvard  33,  Harvard  Apparatus)  are  available.  Syringes  do  not  have  to  be  the  same  size.  Each 
syringe  has  its  own,  computer  controlled  block  so  that  perfusion  rates  can  be  controlled  independently.  An 
open,  perfusion  microincubator  (PDMI-2,  Harvard  Apparatus)  has  been  mounted  on  the  Zeiss  Axiovert.  A 
bipolar  temperature  controller  (TC-202)  is  available  for  use  with  the  PDMI-2.  This  is  a  Peltier  electronic 
instrument  that  delivers  and  adjusts  DC  current  to  the  microincubator  to  control  temperature.  A  "Bath"  type 
thermistor  (BSC-T3,  36K  Ohms  total)  is  used  with  the  PDMI-2.  A  manual  and/or  computer  controlled 
microcapillary  pdfeitioner  (Model  MP-185  Motorized  Huxley- Wall  Type  Micromanipulator  from  Sutter 
Instrument  Co.,  Novato,  CA)  is  available  for  microdissection  work. 
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FIGURE  1 :  (A)  Schematic  of  cells  in  co¬ 
culture  undergoing  perfusion.  Four  fields 
of  view  are  being  monitored  (gray  squares) 
Cell  type  1  on  the  left,  cell  type  2  on  right. 
(B)  Current  VSOM  system  with  four  com¬ 
puter-controlled  syringes  and  an  inverted. 


FIGURE  2:  Microperfusion  chamber  mounted  on  Zeiss  Axiovert. 

(A)  Transmitted  light  condenser  tilted  back  to  show  chamber. 

(B)  Close-up  of  chamber  showing  vacuum  aspirator  (red  base), 
temperature  probe  (blue  base),  and  other  perfusion  input  lines. 


ONE  FIELD  OF  VIEW  IN  A  35mm 
DISH.  MONITOR  CELLS  DURING 
TIME-LAPSE  EXPERIMENT,  RETURN 
TO  THIS  FIELD  OF  VIEW  DAYS  LATER. 


Zeiss  Axiovert  fluorescence  microscope. 


FIGURE  3:  Schematic  of  current  approach  for  linking  early 
responses  to  eventual  cell  fate  on  a  cell-by-cell  basis.  Use  VS 
to  return  repeatedy  to  same  field  of  cells  after  additional  time 
in  incubator.  Cross-hairs  are  etched  into  surface  of  plate  (red) 

MCF-7  CELLS 


FIGURE  4:  Schematic  of  VSOM 
experiment.  (A)  Cell  nuclei  pre¬ 
labeled  with  Hoechst  33342,  begin 
Lyso-Tracker  Red  perfusion  (B)  Cell 
lysosomes  labeled,  begin  calcein-AM 
perfusion  (C)  relabeled  lysosomes. 
Actual  experimental  data  of  this  ex¬ 
periment  shown  in  Figure  5. 
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FIGURE  5:  Actual  experimental  results  for  the  experiment 
depicted  in  Figure  4.  325  cells  were  monitored  in  the  red, 
green,  blue,  and  transmitted  light  channels.  Error  bars  indicate 
+/-  standard  deviation. 
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FIGURE  6:Use  of  multiwell  fluorescence  plates.  (A)  Stage  holder  for  using  plates  on  microscope. 

(B)  Cytofluar  fluorescence  plate  reader  (C)  Loading  plates  in  sterile  hood  using  multichannel  pipetter 
(red  outline)  (D)  Gas  chamber  for  incubator  (front  lid  sealed)  (E)  Lid  open  (F)  Image  of  living  cells  in  one 
well  of  a  plate,  acquired  using  live/dead  stain  lOX  objective.  (All  images  on  page  at  same  magnification.) 


FIGURE  8:  Thapsigargin-induced  generation  of 
mRNA  demonstrated  in  red  channel.  Human  mam¬ 
mary  epithelial  cells  (184B5)  were  stained  with 
LDS-751  prior  to  stimulation  with  thapsigargin. 

(A)  Early  time  point  (40  min)with  mRNA  response 
in  only  a  few  cells  (B)  Later  time  point  (55  min) 
with  more  cells  responding. 


FIGURE  7:  Selected  images  from  a  stress  response,  time-lapse  experiment  on  MCF-7  cells.  Images  A  and  C 
and  B  and  D  are  the  same.  However,  A  and  B  have  the  transmitted  light  image  overlaid  on  the  combined 
red,  green,  and  blue  charmels  which  represent  LysoTracker  Red,  Calcein-AM,  and  Hoechst  33342  fluores¬ 
cence,  respectively.  Cells  were  stressed  by  low  calcium  buffer  and  excessive  UV  illumination.  Lysed  cells 
with  blebbing  membrane  and  no  green  dye  may  be  observed  in  the  image  from  the  later  time  point,  image  B/D. 
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FIGURE  9:  Quantitation  of  BrdU  incorporation  using  dual-stained,  living  cells  and  image-ratio  technique. 
Cells  were  pulse-labeled  with  BrdU  for  1  hr,  then  were  dual-stained  with  Hoechst  33342  and  Sytol6.  Images 
acquired  with  360  and  490nm  excitation,  respectively,  were  ratioed  on  a  pixel  by  pixel  basis  and  color-coded. 
Three  separate  dishes  of  HMECs  (184B5)  in  images  A,  B,  and  C:  (A)  Cells  deprived  of  EGF  for  72hr. 

(B)  Cells  deprived  of  EGF  for  48  hr,  then  received  EGF  24  hrs  ago  (C)  Cells  that  were  never  deprived  of 
EGF  and  were  refed  24  hrs  ago.  Results  agree  with  Stampfer,  et  al.  Exp  Cell  Res.,  208:175-188  (1993). 
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FIGURE  10:  Example  of  nucleus  segmentation  in  tissue 
sections  stained  wiA  DAPI.  (A)  original  (B)  segmentation 
by  hand  (C)  one  of  our  segmentation  algorithms 


FIGURE  1 1  Living  MCF-7  cells  stained  with 
TMRE.  Both  the  plasma  and  motchondrial 
membrane  potential  can  be  calculate  from  this 
image  (Farkas,  et  al.).  63X,  oil  objective  used. 


A 


C 
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FIGURE  12:  These  are  examples  of  three  different  image  segmentation  algorithms  that  are  currently  under 
development.  (A,B)  Nuclei  of  living  184B5  cells  stained  with  Sytol6.  The  same  cells  exhibit  mRNA  staining 
in  another  charmel,  as  shown  in  Figure  8.  (C,D)  Nuclei  of  living  MCF-7  cells  stained  with  Hoechst  33342. 
This  is  the  blue  channel  of  the  cells  seen  in  Figure  7.  (E,F)  Segmentation  of  cells  using  transmitted  light  only. 
Our  goal  is  to  be  able  to  segment  cells  in  the  transmitted  light  channel,  and  thus  avoid  staining  the  cells  in  any 
way.  At  the  present  time,  our  algorithms  are  only  working  with  phase-contrast  images  taken  at  20X.  We  hope 
to  improve  ^s  algorithm  until  we  can  segment  cells  at  lOX,  using  only  brightfield  illumination.  The  use  of 
phase  contrast  objectives  is  not  recommended  for  sensitive  fluorescence  studies  of  living  cells.. 
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Our  goal  in  the  proposed  research  is  to  use  visual  servoing  optical  microscopy  (VSOM)  to 
rapidly  analyze  large  populations  of  living  cells  that  have  been  freshly  derived  from  human  breast 
cancer  tissue  biopsies.  We  believe  the  rapid  analysis  of  dynamic  physiological  responses  on  a 
patient-by-patient,  tumor-by-tumor,  and  cell-by-cell  basis  will  provide  critical  information  for 
individualized  cancer  chemotherapy.  We  have  assembled  a  multidisciplinary  team  with  expertise  in 
engineering,  computer  sciences,  digital  imaging,  cell  biophysics,  and  cancer  biology.  The;  successful 
accomplishment  of  the  research  objectives  proposed  here  would  represent  an  important  advance  in  in 
vitro  chemosensitivity  testing  and  would  represent  a  large  step  toward  our  ultimate  goal  of 
chemotherapy  regimens  tailored  to  the  individual  breast  cancer  patient.  Visual  servoing  (VS)  is  a  term 
from  the  field  of  robotic  vision.  In  our  case,  it  refers  to  the  dynamic  manipulation  of  experimental 
parameters  based  on  analysis  of  digital  image  content.  A  VSOM  fluorescence  assay  consists  of  (i) 
real-time  observations  of  individual  cell  responses  to  a  series  of  applied  stimuli  (ii)  automatic,  and 
intelligent  adjustment  of  experimental  parameters  to  achieve  experimental  goals.  Our  immediate  goal 
is  to  develop  a  wide  range  of  VSOM  fluorescence  assays  that  rapidly  (1-8  hrs)  produce  detailed  and 
distinct  physiological  fingerprints  that  are  highly  predictive  of  an  individual  patient's  response  (both 
normal  and  tumor  cell  response)  to  a  particular  anti-cancer  drag,  particular  combination  of  drags,  or 
particular  sequence  of  drag  exposures.  An  ideal  VSOM  fluorescence  assay  does  not  irreversibly 
perturb  living  cells,  and  it  allows  one  to  make  predictions  of  drag  sensitivity  based  on  specific 
physiological  characteristics  of  individual  cells. 

Hypothesis:  We  believe  our  technology  will  allow  us  to  rapidly  (i)  develop  new  in  vitro  fluorescence 
assays  that  can  be  used  to  predict  chemosensitivity  (ii)  develop  selective  culture  conditions  for  specific 
subpopulations  of  cells,  i.e,  primary  culture  tumor  cells  (iii)  demonstrate  a  new  paradigm  in  in  vitro 
chemosensitivity  testing.  This  new  paradigm  is  visual  servoing  for  knowledge-based  interrogation  of 
individual  living  cancer  cells. 
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Several  companies  have  recently  been  formed  to  pursue  the  idea  of  testing  each  individual  breast 
cancer  patient’s  tumor  cells  in  a  test  tube  or  dish  outside  the  patient’s  body 
(http://www.oncodocs.com/laboratories/laboratories.html).  Such  tests  are  performed  on  each  patient’s 
cancer  cells  in  order  to  determine  the  best  drug  regimen  for  that  particular  patient.  This  type  of  testing 
is  referred  to  as  “in  vitro’’  chemosensitivity  testing  or  “predictive  testing”.  The  goal  of  our  proposed 
research.is  to  demonstrate  the  feasibility  of  a  new  class  of  in  vitro  drug  response  and  chemosensitivity 
assays.  Our  objectives  are  to  develop  a  computer  controlled  microscope  (a  visual  servoing  optical 
microscope,  or  VSOM)  that  can  rapidly  and  automatically  test  a  patient’s  cancer  cells  prior  to  the 
selection  of  an  anti-cancer  drug  regimen.  We  believe  our  instrument  will  not  only  be  useful  for  the 
ultimate' application  of  such  tests;  it  will  also  be  useful  for  the  development  of  additional  tests  that  are 
more  sensitive  and  precise.  Our  goal  is  to  build  the  instrument  and  perform  several  demonstrations 
that  will  get  other  scientists  and  funding  agencies  excited  about  our  new  approach.  Several  aspects  of 
our  approach  are  unique.  We  have  combined  sophisticated  robotic  vision  techniques,  digit^  video 
microscopy  and  biophysics  to  actually  observe  the  responses  of  hundreds  of  living  cells  in  a  variety  of 
colors.  The  different  colors  represent  glowing  fluorescent  tracers  within  the  living  cells.  In  these 
assays,  computer  controlled  syringes  inject  various  substances  into  a  temperature  controlled  cell 
chamber.  We  are  currently  observing  the  single  cell  responses  of  up  to  500  cells  in  three  or  four 
different  colors.  In  addition,  we  can  observe  the  traditional,  transmitted  light  image  of  all  the  cells.  In 
one  model  system,  we  propose  to  study  normal  human  mammary  epithelial  cells.  In  a  second  model 
system,  (drug  sensitive  and  multi-drug  resistant  human  breast  cancer  cell  lines;  MCF-7  cells)  our 
results  indicate  that  current  fluorescence  assays  for  multidrug  resistance  are  indeed  in  need  of 
improvement.  We  will  attempt  to  develop  improved  versions  of  these  tests  in  this  three  year  grant. 
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1 .  Indicate  the  one  (1)  BCRP  priority  issue  targeted  best  by  this  application  (see 
Section  2): 

Q  Health  Care  Delivery  and  Health  Policy  Q  Socio-cultural,  Behavioral,  and  Psychological  Issues 

Q  Biology  of  the  Normal  breast  Q  Prevention 

□  Etiology  Q  Earlier  Detection 

X  Innovative  Treatment  Modalities  Q  Pathogenesis 


2.  Indicate  the  award  type  for  this  application: 

Q  SPRC  Exploratory  Award  X  SPRC  Full  Research  Award 

□  TRC  Pilot  □  TRC  Full  Research  Award 

□  IDEA-I  □  IDEA-II 

□  New  Investigator  □  Postdoctoral  Fellowship 

□  Training  Program  □  RFA 


Discuss  how  the  priority  issue  and  award  type  are  targeted  by  the  research  project  and  how  they  are 
responsiye  to  the  mission  of  the  BCRP  in  reducing  impact  of  breast  cancer  in  Caiifornia.  Use  one 

(1)  continuation  page  (5 A),  as  needed. _ 

The  title  of  our  proposed  research  is  “Predictive  Testing  for  the  Individual  Breast  Cancer  Patient”.  In  the 
proposed  research,  we  are  targeting  Priority  Issue  7,  “Innovative  Treatment  Modalities:  Search  for  a  Cure”. 
Specifically,  we  are  proposing  a  new,  rational  therapeutic  approach  for  tailoring  drug  combinations  to 
individuals  using  a  novel  approach  to  in  vitro  chemosensitivity  testing.  We  have  constructed  a  Visual 
Servoing  Optical  Microscope  (VSOM),  and  will  use  it  to  design,  execute,  and  archive  the  results  of  VSOM 
fluorescence  assays.  This  is  a  new  technology  and  a  new  approach  to  testing  living  cells.  An  important 
feature  of  VSOM  fluorescence  assays  is  the  ability  of  the  system  to  interact  with  an  ensemble  of  living  cells 
in  an  intelligent  fashion,  during  the  course  of  each  assay.  This  approach  combines  sophisticated  robotic 
vision  techniques  with  biophysical  models  of  single-cell  responses  to  applied  stimuli.  A  cuirently  funded 
breast  cancer  researcher  (Dr.  D.E.  Callahan)  with  extensive  experience  with  biophysical  studies  of  cancer 
cells  will  be  working  with  a  computer  vision  expert  (Dr.  B.  Parvin).  Thus,  we  have  chosen  to  apply  for  a 
Scientific  Perspectives  Research  Collaboration  (SPRC)  Awaird.  This  will  allow  a  breast  cancer  researcher 
(Dr.  Callahan)  an  excellent  opportunity  to  continue  to  work  with  Dr.  Parvin  and  continue  to  integrate  him 
into  the  ongoing  breast  cancer  research  in  our  Life  Sciences  Division.  We  have  new  tools  and  theories  to 
apply  to  the  field  of  breast  cancer  research.  The  technology  revolution  that  has  produced  robotic  vision, 
powerful  computers,  massive  storage  capabilities,  object-oriented  databases,  and  Internet  connectivity  has 
not  yet  been  fully  applied  to  current  problems  in  breast  cancer  research.  What  is  required  are  scientists  who 
are  willing  to  bridge  the  gaps  between  computer  science,  molecular  and  cell  biology,  DNA  and  gene 
technologies,  medicine,  physiology,  chemistry,  and  biophysics. 

Our  team  of  scientists  at  Lawrence  Berkeley  National  Laboratory  (LBNL)  hsis  unique  resources,  expertise 
and  experience.  We  often  work  together  in  large  multi-disciplinary  teams.  Phase  1  of  our  VSOM  project  is 
nearly  complete.  Drs.  Parvin  and  Callahan  are  midway  through  a  2-vear  Department  of  Defense  breast 
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cancer  IDEA  grant,  “Visual  Servoing  for  Optimization  of  Anti-Cancer  Drug  Uptake  in  Human  Breast 
Cancer  Cells”.  In  this  grant,  we  have  constructed  a  VSOM  and  are  in  the  process  of  demonstrating 
a  new  approach  to  disease  prevention  that  will  identify  and  characterize  drug  sensitive  and  drug 
resistant  tumor  cells  on  a  patient-bv-patient  and  tumor-bv-tumor  basis.  We  believe  we  are  responsive 
to  the  SPRC  award  type.  We  have  a  strong  commitment  to  breast  cancer  research  ^ecause  it  is  the 
focus  of  our  Division  and  our  Department.  There  are  many  future  opportunities  for  interaction  with 
the  other  breast  cancer  researchers  in  our  building  and  this  will  be  our  focus  for  the  foreseeable  future 
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•  This  information  is  used  for  the  programmatic  review  (see  Section  3). 

•  You  may  add  one  continuation  page  (page  6A)  to  this  form. 

Discuss  how  your  project  addresses  the  following  BCRP  programmi&tic 
criteria 

Multidisciplinary  Approach:  (incorporation  of  researchers  and  methods 
from  several  fields) 

We  have  a  truly  multidisciplinary  team  and  meet  the  additional  criteria  for  a  full  SPRC 
grant.  We  have  obtained  pilot  data,  established  a  solid  organizational  infrastructure,  and 
developed  protocols  (administrative,  financial,  etc.)  for  collaboration  between  Computer 
Scientists  and  Biologists.  Many  obstacles  to  such  collaboration  have  been  overcome 
during  the  last  several  years.  For  example,  it  is  difficult  to  compete  with  the 
overwhelming  demand  for  computer  scientists  of  the  highest  caliber,  as  may  be  seen  by 
their  cost  to  the  project.  In  addition,  computer  scientists  and  biologists  at  LBNL  are 
physically  separated  on  a  large  campus  and  have  very  different  administrative  structures, 
cultures,  and  vocabularies.  Even  their  computer  platforms  (Sun/UNIX  vs.  PCAVindows) 
are  quite  different  and  can  hinder  collaborations.  We  have  a  proven  track  record  of 
collaboration.  Drs.  Parvin  (Information  and  Computing  Sciences  Division,  ICSD)  and 
Callahan  (Life  Sciences  Division,  LSD)  have  spent  over  5  years  introducing  the  animal, 
cell  and  cancer  biologists  in  our  Division  to  multi-channel  (multi-color)  digital  imaging 
fluorescence  microscopy.  The  Life  Sciences  Microscope  Resource  (LSMR),  under  the 
direction  of  Dr.  Mary  Helen  Barcellos-Hoff  (our  colleague  and  consultant  on  this  grant) 
has  helped  cancer  biologists  move  from  selective  photographic  documentation  of 
biological  phenomena  in  cells  and  tissue  to  quantitative  biology  with  non-biased 
sampling  methods  and  statistical  analysis  of  digital  image  information.  In  addition,  the 
computer  scientists  and  biophysicists  have  come  to  appreciate  the  most  complex  and 
most  pertinent  biological  systems  (tissue  and  animals)  and  have  learned  to  focus  more 
intently  on  the  rapid  production  of  important  biological  results.  These  interactions  have 
produced  several  fruitful  collaborations  between  our  Divisions,  several  of  which  are 
currently  in  progress.  The  Parvin/Callahan/Barcellos-Hoff  research  partnership  obtained 
funding  using  the  LSMR  as  a  method  of  collaboration  and  has  now  moved  on  to  a  second 
level  of  collaboration.  The  second  phase  involves  shared  equipment  in  the  LSMR, 
compatible  software  packages  and  image  formats,  custom-written  software,  graphical 
user  interfaces,  image  libraries,  databases  and  seamless  network  connnectivity.  This 
research  collaboration  is  underway,  and  is  taking  place  on  different  computer  platforms 
arid  operating  systems. 

Translational  Potential:  (potential  to  be  used  for  direct  and  immediate 
impact  on  breast  cancer) 

Several  companies  currently  perform  in  vitro  chemosensitivity  testing.  Such  tests  are 
being  performed  on  patient’s  biopsied  cancer  cells  in  order  to  determine  the  best  drug 
regimen  for  that  particular  patient.  Also,  health  insurance  companies  are  beginning  to 
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pay  for  this  testing  (See  http://www.oncodocs.com/laboratories/laboratories.html  and 
Scientific  American,  Feb.  1999  article).  We  plan  to  demonstrate  the  application  of 
VSOM  assays  in  traditional  multiwell  plate  fluorescence  assays  in  order  to  verify  the 
translational  potential  of  information  obtained  using  a  VSOM.  It  should  be  possible  to 
replicate  many  of  our  timed  agent  exposures  and  buffer  exchanges  manually  in  rrtultiwell 
plates.  Thus,  the  improved  tests  would  be  widely  available  immediately  and  our 
customized  hardware  would  not  be  required.  However,  we  believe  that  a  VSOM  will  be 
very  useful  for  executing  fluorescence  assays,  not  just  developing  them.  Thus,  we  have 
filed  an  LBNL  Patent  Disclosure:  “A  Method  for  Knowledge-Based  Discovery  and 
Optimization  of  Differences  Between  Cell  Types”  Callahan,  D.E.,  and  Parvin,  B. 
September  9,  1999.  We  are  very  interested  in  working  with  commercial  companies  and 
making  this  technology  available  as  soon  as  possible. 

In  addition,  any  VSOM  information  we  can  obtain  regarding  selective  media  conditions 
for  the  propagation  of  primary  human  breast  cancer  cells  would  also  be  of  immediate  and 
practical  value  to  researchers,  new  drug  developers,  and  in  vitro  chemsensitivity  testing 
companies. 

Focus  on  Underserved  Populations:  (potential  to  reduce  disparities  in 
incidence  and  treatment) 

A  flexible  chemotherapeutic  regimen  designed  for  each  individual  breast  cancer  patient, 
rather  than  the  representative  “average”  population  would  reduce  disparities  in  the 
treatment  of  breast  cancer.  Average  population  results  will  often  work  better  for 
majorities  rather  than  minorities.  “Minority”  in  this  sense  can  mean  any  minority, 
including  women  who  are  not  sensitive  or  are  hypersensitive  to  standard  drugs,  or 
women  with  a  unique  genetic  disposition  or  other  complicating  factor  that  will  never  be 
represented  in  a  large  study. 


Annual  Report:  Callahan,  Parvin  Page  101 


Title: 

Institution: 

PI: 

Position  of  PI: 

Co-PI: 

Address: 


Telephone: 

FAX: 

Email: 

Requested  funding: 

Use  of  human  subject: 

Use  of  vertebrate  animals: 


Visual  Servoing  for  Knowledge-Based 
Interrogation  of  LivingHuman  Cancer  Cells 

Lawrence  Berkeley  National  Laboratory 
Bahram  Parvin 
Staff  Scientist 

Dan  Callahan  and  Ge  Cong 
M.S.  50B-2239 

Lawrence  Berkeley  National  Laboratory 

Berkeley,  CA  94720 

510-486-6203 

510-486-6363 

parvin@george.lbl.gov 

Year  1  ($224,777),  Year  2  ($218,608),  Total  ($443,385) 

None 

None 


Signature: 


Date: 


Annual  Report:  Callahan,  Parvin  Page  102 


1  Introduction 


Visual  servoing  [15]  refers  to  the  dynamic  manipulation  of  experimental  parameters  (in  this  case,  th 
concentrations  of  anti-cancer  drugs  or  other  agents)  ba^ed  on  the  analysis  of  image  content.  Our  go; 
is  to  use  visual  servoing  optical  microscopy  (VSOM)  to  rapidly  analyze  large  popufations  of  living  an 
cultured  human  cells  that  have  been  freshly  derived  from  cancer  tissue  biopsies.  We  beheve  the  rapi 
analysis  of  dynamic  physiological  responses  on  a  patient-by-patient,  tumor-by-tumor,  and  ceU-by-ce 
basis  will  provide  critical  information  for  individualized  cancer  chemotherapy.  We  have  assembled 
multidisciplinary  team  with  expertise  in  engineering,  computer  sciences,  imaging,  cell  biophysics,  an 
cancer  research.  The  successful  accomplishment  of  the  research  objectives  proposed  here  would  represent 
an  important  achievement  in  biomedical  engiireering.  A  VSOM  fluorescence  assay  consists  of  (i)  real¬ 
time  observations  of  individual  cell  responses  to  a  series  of  apphed  stimuli  (ii)  automatic,  and  intelligent 
adjustment  of  experimental  parameters  to  achieve  experimental  goals.  An  ideal  VSOM  fluorescence 
assay  does  not  irreversibly  perturb  living  cells,  and  it  allows  one  to  make  correlations  between  specific 
physiological  characteristics  of  a  cell  and  the  subsequent  sensitivity  of  the  cell  to  anti-cancer  drugs,  or 
other  agents.  Our  long  range  goal  is  to  develop  a  wide  range  of  VSOM  fluorescence  assays  that  rapidly 
(1-6  hrs)  produce  detailed  physiological  VSOM  “fingerprints”  of  a  large  population  of  cells  on  a  cell- 
by-cell  basis.  We  believe  our  technology  will  allow  us  to  develop  specific  VSOM  assays  that  are  highly 
predictive  of  an  individual  patient’s  response  (tumor  and  normal  cell  chemosensitivity)  to  a  particular 
anti-cancer  drug,  or  combinations  of  drugs.  We  also  believe  our  technology  will  allow  us  to  rapidly 
develop  optimized  culture  conditions  for  specific  subpopulations  of  cells.  There  will  be  three  phases  in 
the  proposed  research,  and  three  funded  programs  will  be  leveraged  to  meet  the  program  goals. 

Phase  1:  Initial  construction  of  a  database  consisting  of  baseline  data  collected  during  VSOM  assays 
that  target  specific  physiological  responses  and  specific  cellular  compartments  in  model  ceU  systems. 

Phase  2:  Refinement  of  this  database  by  modifying  experimental  parameters  during  real-time  ex¬ 
periments  until  the  assay  reaches. the  desired  level  of  predictive  power  in  model  cell  systems. 

Phase  S:  Intelligent,  knowledge-based,  real-  time  analysis  of  unknown,  paired  specimens  (e.g.,  phys¬ 
ically  separated  normal  and  tumor  cell  populations).  This  will  require  automatic  application  of  the 
proper  VSOM  assays  in  the  proper  order,  using  the  appropriate  experimental  parameters.  The  end 
result  of  such  an  analysis  will  be  a  report  that  compares  the  effects  of  different  drugs  (or  other  agents) 
on  the  two  populations.  These  comparisons  wiU  be  based  on  proliferation,  apoptosis,  or  any  other 
pertinent  cellular  responses  that  can  be  monitored  during  the  experiment. 

Visual  servoing  aims  at  automated  manipulation  of  instrument  control  parameters  based  on  auto¬ 
matic  visual-scene  interpretation.  A  simple  example  of  visual  servoing  is  autofocusing  for  an  optical 
microscope,  where  the  microscope  objective  is  moved  automatically  along  the  Z  direction  to  compensate 
for  defocus.  In  this  context,  defocus  is  quantified  and  the  Z-axis  is  moved  under  computer  control  for 
optimum  focus.  A  more  complex  example  includes  drift  compensation  or  motion  detection  as  a  function 
of  external  stimuh  for  dynamic  experiments  [12].  This  system  (currently  in  use  at  LBNL)  not  only 
detects  motion  in  the  scene,  but  also  records  and  logs  pertinent  attributes,  e.g.  size,  shape,  response, 
direction  of  motion.  This  information  is  then  compared  to  attributes  of  similar  events  that  are  stored  in 
a  database.  Another  benefit  of  this  level  of  automation  is  that  these  dynamic  experiments  can  be  con¬ 
ducted  remotely  over  the  wide  area  network  [15].  Automatic  drift  compensation  provides  a  stationary 
view  of  the  experiment  and  eliminates  the  need  for  high  speed  network  and/or  quality  of  service. 

The  proposed  research  requires  close  collaboration  between  biologists  and  computer  scientists.  The 
investigators,  listed  in  this  proposal,  have  a  proven  track  record  of  scientific  collaboration  [11,  15]. 
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From  a  computer  science  perspective,  the  following  components  are  needed  to  achieve  the  research 
objectives:  (1)  computer  control  of  the  microscope  and  syringes,  (2)  segmentation  of  cells  imaged  on 
single  or  multiple  channels,  (3)  annotation  of  the  assay  used  in  a  particular  experiment,  (4)  automated 
annotation  of  cell  responses  as  a  function  of  a  schedule,  (5)  offline  correlation,  clustering,  and  learning 
techniques  that  compare  the  differences  between  two  cell  types  as  a  function  of  an  assay,  and  (6)  online 
learning  techniques  for  dynamic  control  and  estimation  of  a  policy  (schedule)  to  elicit  distinct  responses 
from  two  physically  separated  ceU  types.  The  interaction  between  various  system  components  are  shown 
in  Figure  1.  These  requirements  leverage  three  research  programs  performed  under  other  grants  that 
are  funded  through  DOE-MICS  {DOE2000  Material  and  MicroCharacterization  CoUaboratory),  DOE- 
OBER  {An  Imaging  Bioinformatic  System  for  Studying  the  effect  of  Low  Dose  Radiation),  and  a  DOD 
Breast  Cancer  research  grant  (Visual  Servoing  for  Anti  Cancer  Drug  Uptake).  The  first  grant  provides 
the  basis  for  remote  collaborative  microscopy,  seamless  data  acquisition,  and  instrument  control.  The 
second  grant  provides  the  imaging  bioinformatic  component  for  annotating  data  and  images  and  the 
needed  core  technology  for  segmentation  of  cells.  The  third  grant  provides  a  platform  for  VSOM  that 
will  be  available  at  the  end  current  fyscal  year.  Our  current  VSOM  grant  (funded  by  the  DOD  Breast 
Cancer  Research)  does  not  include:  (1)  database  component  and  population  of  that  database  with 
ceU  responses,  (2)  learning,  clustering,  and  data  mining  in  that  database,  (3)  real-time  analysis  and 
manipulation  of  physically  separated  population  of  cells,  (4)  segmentation  of  cells  using  data  fusion 
technique  (images  acquired  on  multiple  channels),  (5)  incorporation  of  large  number  of  syringes,  and 
(6)  study  of  mitochondrial  physiologies.  We  are  confident  that  by  coupling  this  proposal  with  our 
currently  funded  programs,  a  new  biomedical  technology  can  be  demonstrated  so  that  drug  therapy  can 
be  tailored  to  an  individual  patient. 

Section  2  provides  an  overview  of  the  system  architecture.  Section  3  provides  the  review  of  core 
technology  for  segmentation  of  cells.  Section  4  introduces  two  classes  of  learning  in  our  system.  Section 
5  summarizes  the  experimental  protocol. 

2  System  Architecture 

The  current  trend  in  telepresence,  distributed  control,  and  computation  research  is  to  bring  experts 
and  facilities  together  from  geographically  dispersed  locations  [15,  18,  19].  The  natural  evolution  of  this 
research  is  to  construct  a  scalable  system  for  collaboration  and  to  leverage  either  legacy  or  new  compu¬ 
tational  toolkits  to  support  novel  scientific  applications  based  on  capabilities  in  real-time  control  and 
steering,  visualization,  and  simulation.  Under  the  DOE2000  program,  the  Berkeley  team  has  developed 
a  collaborative  system  [16]  for  remote  microscopy.  The  software  architecture  for  this  Enterprise-wide 
information  service  is  shown  in  Figure  2.  Our  system  uses  an  extensible  object  oriented  framework  so 
that  apphcations  can  be  rapidly  assembled,  maintained,  and  reused.  These  objects  may  reside  on  any 
host  and  can  be  listed,  queried,  and  activated  in  the  system. 

Our  system  consists  of  three  service  categories  that  interact  with  the  object  request  broker  (ORB): 
Instrument  Services  (IS),  Exchange  Services  (ES),  and  Computational  Services  (CS).  These  core  services 
are  specified  with  an  interface  that  is  language  independent.  Instrument  Services  provide  a  scalable 
means  of  instrument  control  and  interaction  through  an  object  hierarchy.  Exchange  Services  provide  a 
scalable  mean  to  collect  data  and  push  them  to  users  or  other  appUcation  software,  e.g.  analytical  or 
storage  facilities.  Computational  Services  provide  means  for  distributed  image  analysis,  visualization, 
simulation,  etc.  The  important  issue  is  that  the  architecture  shown  in  Figure  2  is  distributed.  For 
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example,  complex  image  segmentation  servers  are  registered  with  a  symmetric  multiprocessing  system, 
where  simple  instrument  control  are  registered  with  the  local  host  near  the  microscope.  All  objects  in 
this  architecture  are  remotely  accessible.  As  a  result,  other  investigators  will  be  able  to  conduct  an 
experiment  and  view/share  their  results. 

3  Segmentation 

Automatic  dehneation  of  cell  nuclei  and  cytoplasm  is  an  important  step  in  mapping  ceU  responses  into 
specific  cellular  compartments.  This  computational  coihponent  interacts  with  the  Instrument  Services 
to  acquire  image  data  from  one  channel  or  multiple  channels  (by  rotating  the  filter  wheel  or  opening 
and  closing  shutters).  A  brief  review  of  the  segmentation  tool  kits  is  provided  [3,  4]. 

3.1  Segmentation  of  Nuclei  from  Single  Channel 

Presently,  VSOM  supports  Hough  transform  to  detect  and  track  circular  objects  (cells  or  nuclei).  The 
technique  works  well  for  both  transmitted  and  fluorescence  light  images.  Under  our  current  DOE-OBER 
grant,  we  have  developed  a  new  method  to  localize  clumped  nuclei  in  a  tissue  section.  This  method 
focuses  on  delineation  of  individual  nuclei  that  are  observed  with  an  epi-fluorescence  microscope.  The 
nuclei  of  interest  reside  in  a  ring  of  cells  that  surround  a  particular  type  of  capillary  in  the  tissue.  The 
intent  is  to  build  the  necessary  computational  tools  for  large  scale  population  studies  and  hypothesis 
testing.  These  nuclei  are  clumped  together,  thus,  making  quick  delineation  infeasible.  An  example 
is  shown  in  Figure  3(a)(b).  Previous  efforts  in  this  area  have  been  focused  on  thresholding,  local 
geometries,  and  morphological  operators  for  known  cell  size  [20,  22].  Others  have  focused  on  an  Optimal 
Cut  Path  that  minimizes  a  cost  function  in  the  absence  of  shape,  size,  or  other  information  [10,  8,  26]. 
These  algorithms  developed  for  tissues  will  be  useful  when  segmenting  adjacent  cultured  cells  into 
nuclear  and  cytoplasmic  region.  ' 

Our  current  research  focuses  on  extraction  of  partial  information  corresponding  to  step  and  roof 
edges  and  then  group  them  through  geometric  reasoning  [3,  4].  The  intent  is  to  construct  a  partitioning 
that  is  globally  consistent.  A  list  of  step  edges  is  constructed  by  thresholding  raw  images,  recovering 
corresponding  boundaries,  and  localizing  concave  corners  from  their  polygonal  approximation.  These 
corners  provide  possible  cues  to  where  two  adjacent  nuclei  may  touch  each  other.  Thresholding  separates 
big  clumps  consisting  of  several  nuclei  squeezed  together,  but  the  boundaries  between  adjacent  nuclei 
cannot  be  detected  since  they  have  higher  intensities  than  background.  However,  the  partial  boundaries 
between  adjacent  nuclei  can  be  recovered  by  extracting  crease  segments  [9,  24,  23],  as  shown  in  Figure 
3(d).  These  crease  segments  correspond  to  trough  edges  (positive  curvature  maxima),  but  false  creases 
may  also  be  extracted  in  the  process.  Nevertheless,  dealing  with  noise  and  erroneous  segments  is  a 
higher-level  process  that  is  handled  during  the  grouping  process.  Our  system  generates  a  number  of 
hypotheses  for  possible  grouping  of  the  boundary  segments.  A  unique  feature  of  this  system  is  in 
hyperquadric  representation  of  each  hypothesis  and  the  use  of  this  representation  for  global  consistency. 
The  main  advantage  of  such  a  parameterized  representation  (as  opposed  to  polygonal  representation) 
is  compactness  and  better  stability  in  shape  description  from  partial  information.  In  this  fashion,  each 
step-edge  boundary  segment  belongs  to  one  and  only  one  nucleus  while  each  trough-edge  boundary 
segment  is  shared  by  two  and  only  two  nuclei.  These  initial  hypotheses  and  their  localized  inter¬ 
relationship  provides  the  basis  for  search  in  the  grouping  step.  This  is  expressed  in  terms  of  an  adequate 
cost  function  and  minimized  through  dynamic  programming.  The  final  result  of  this  computational  step 
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is  then  shown  to  an  operator  for  verification  and  elimination  of  false  alarms.  An  example  is  shown  in 
Figure  4,  where  comparison  to  manual  segmentation  is  shown. 

3.2  Segmentation  of  Cells  from  Multiple  Channels  i). 

Another  tedious  type  of  data  collection  is  in  the  analysis  of  multichannel  images.  In  this  scenario,! 
a  sample  is  imaged  with  different  dyes  and  filters  to  accentuate  different  components  of  the  cells. | 
Here,  the  system  must  be  able  to  analyze  several  channels  of  data  simultaneously,  select  a  specific* 
location  in  the  cytoplasmic  region,  and  make  repeated  measurements.  An  efficient  approach  to  fusing 
of  images  based  on  a  Bayesian  framework  has  been  developed.  This  approach  can  also  be  extended  to 
3D  (with  confocal  microscopes)  as  demonstrated  by  our  earlier  work  [14].  The  purpose  of  data  fusion  is 
complementary  processing  of  different  modalities  for  segmentation  and  labeling.  From  this  perspective, 
the  segmentation  procedure  should  label  each  pixel  in  the  data  volume  accordingly.  However,  there  are 
a  number  of  ambiguities  that  can  complicate  the  labeling  process.  These  ambiguities  can  arise  from 
purely  local  processing  and  the  absence  of  any  high  level  feedback.  The  sources  for  the  ambiguities 
include  corruption  of  data  by  noise,  performance  limitation  of  algorithms  for  extracting  local  features, 
and  existence  of  nonessential  features  that  impede  the  labeling  task.  One  aspect  of  region  segmentation 
is  in  estimating  the  average  intensity  of  each  region.  This  is  done  by  analyzing  the  histogram.  The 
initial  position  of  the  peaks  in  the  histogram  are  approximated  and  then  refined  with  least  square 
approximation.  The  next  step  of  the  computational  process  is  to  use  these  peaks  as  cluster  centers 
to  enforce  local  consistency  in  the  image  space.  Here,  a  Bayesian  framework  is  used  to  label  images 
based  on  their  multiple  channel  information.  Specifically,  we  use  Bayesian  hierarchical  model  with  three 
levels  of  hierarchy.  The  first  level  is  a  model  for  the  underlying  classification,  Z.  Under  ideal  condition, 
observing  Z  under  M  different  modalities  (here,  there  are  three  fluorescence  images  corresponding  to 
blue,  green,  and  red  fluorescence  emission,  respectively)  specifies  an  ideal  representation,  Xj,  of  the  data 
that  are  corrupted  by  different  types  of  noise  in  the  imaging  system.  The  third  level  of  the  hierarchy 
correspond  to  the  actual  observation  of  data  ¥{.  The  first  level  uses  Markov  Random  Field  (MRF) 
with  prior  probability  density  function  in  the  form  of  Ising  model  with  a  positive  parameter  /?.  This 
parameter  encourages  cooperation  among  nearby  pixels.  In  a  Bayesian  framework,  /?  may  assume  prior 
ignorance  about  the  scene  content.  The  second  level  models  X  as  conditionally  independent  given  Z, 
i.e.,  P(Xi,  =  P{Xi\Z)...P{Xm\Z).  This  is  reasonable  since  if  a  classification  is  labeled  as 

cytoplasm  then  the  ideal  response  of  the  green  channel  (cytoplasm)  and  the  blue  channel  (nuclei)  will 
not  affect  each  other.  Finally,  the  third  level  of  hierarchy,  Yi,  is  modeled  as  a  Gaussian  distribution.  As 
a  result,  Yi  will  be  a  locally  blurred  representation  of  Xi.  The  full  model  can  be  written  as 

P(Z,Xi\Yi}  =  llP{Yi\Z)l[P{Xi\Z)P{Z)  (1) 

We  use  maximum  a  posteriori  (MAP)  estimates  to  obtain  Xi  and  Z.  However,  since  MAP  computation 
is  inherently  infeasible,  we  use  numerical  approximation  based  on  iterative  conditioning  modes  (ICM) 
[1]  to  find  the  local  optimum.  An  example  of  three  channel  fluorescence  image  processing  is  shown  in 
Figures  5. 
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4  Learning  from  cell  responses  ; 

There  will  be  two  categories  of  learning  in  our  system  corresponding  to  offline  and  online  learning.  Offline] 
learning  is  performed  when  a  schedule  for  an  assay  is  designed  and  entered  into  the  system  control  panelj 
(its  GUI).  Massive  amounts  of  data  are  then  collected  as  the  system  records  the  response  of  each  cell  to! 
VSOM'assay  stimuli.  On  the  other  hand,  online  learning  attempts  to  illicit  a  certain  cell  response  .based! 
on  historical  data  (in  the  archive)  and  current  summary  of  cell  responses  uirder  observation...  These^ 
learning  techniques  have  a  common  thread  in  that  they  try  to  cluster  and  learn  behavior  in  a  multi  ■  '  ' 
dimensional  space  as  measured  by  fluctuations  in  the  fluorescence  intensity  emanating  from  differenif’  ' 
cellular  compartments  as  a  result  of  applied  stimuU  (compounds  being  injected  into  the  cell  chamber)! 

AU  measurements  are  made  relative  to  the  system’s  initial  state.  Formally,  the  observed  intensity 'is^ a  ' 
multivariable  function  defined  as 


=  fiCellType^Dye^  [Dye\^  Channel^ 


dl  dFlow, 


dV  dt 


ST) 


Where  I  is  the  mean  fluorescence  intensity  observed  in  the  cytoplasm  at  time  t.  Dye  =  {a  fluorescence  dye  etc. 
Channel  =  {Red,  Green,  Blue,  etc.},  T  is  temperature,  and  corresponds  to  changes  in  the  flow  of 

compounds  (e.g.,  Nigericin,  Valinomycin,  plain  buffer)  being  injected.  An  Schema  wiU  be  developed  to 
capture  this  time  varying  information  to  store  the  observations  into  a  database  for  subsequent  knowledge 
discovery.  Each  experimental  cycle  represents  a  possible  schedule  in  an  assay  {It  for  t  =  0  ....  n)  that 
generates  a  fingerprint  for  each  observed  cell.  These  individual  responses  provide  a  compact  representa¬ 
tion  of  intra-cellular  activities  in  the  image  space.  This  information  wiU  be  stored  and  then  visualized 
per  ceU,  population  of  cells,  cell  types,  etc. 


4.1  Offline  Learning 

Statistical  techniques  to  track  and  to  form  clusters  on  the  basis  of  their  state  and  action  wiU  be  de¬ 
veloped.  We  will  use  entropy  minimization  and  a  numerical  iterative  hierarchical  clustering  (NIHC) 
algorithm.  The  NIHC  starts  with  randomly  assigning  data  to  clusters  in  a  B-tree  structure  and  iter¬ 
atively  reducing  the  total  Gaussian  eirtropy  of  the  tree.  Additionally,  K-mean  approximation  will  be 
used  for  state  approximation.  Co-occurrence  statistics  of  the  dynamic  labels  over  aU  sequence  patterns 
will  then  be  calculated  to  establish  an  equivalence  class. 

Alternatively,  a  ceU  response  and  its  corresponding  attributes  (dynamic  annotations)  generates  a 
representation  within  a  sequence.  This  representation  can  then  be  used  to  construct  joint  co-occurrence 
statistics  in  that  sequence.  These  statistics  can  be  used  to  create  a  binary  tree  classification  of  the 
representation.  The  main  utiUty  of  tliis  method  is  in  detecting  and  classifying  correlated  sequences. 
AdditionaUy,  once  a  model  is  estabUshed,  co-occurrence  statistics  provides  a  mean  to  detect  outUers. 


4.2  Online  Learning 

The  intent  is  to  illicit  a  certain  response  in  order  to  differentiate  between  two  cell  populations.  This  is 
often  referred  to  as  reinforcement  learning  in  the  Uterature,  which  may  be  modeled  as  a  markov  decision 
process  (MDP).  A  particular  MDP  is  defined  by  its  state  (current  observed  intensity  and  its  gradient)  and 
action  set  (concentration  of  compounds  in  the  ceU  chamber  as  dictated  by  their  corresponding  flow  rate). 
The  reward  in  such  a  system  can  be  measured  as  the  global  intensity  difference  and  its  episodic  gradient 
between  two  ceU  population  types.  Furthermore,  the  transition  probabilities  (of  markov  decision  process) 
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can  be  computed  from  observations  made  from  the  same  assay  in  the  database.  These  transiiidn 
probabilities  provide  the  enabling  technology  for  an  optimal  policy  (schedule).  In  general,  such  a  policy 
can  be  expressed  as  dynamic  programming,  Monte  Carlo  methods,  and  temporal  difference  learning.. 
Dynamic  programming  requires  a  model  of  ceU  response.  Such  a  model  can  be  hypothesized  from  the; 
database.  Presumably,  as  the  database  expands,  models  may  become  more  accessible.  Monte  Carlo 
methods  do  not  require  a  model,  but  they  are  not  suitable  for  step-by-step  incremental  computation. 

On  the  other  hand,  temporal  methods  require  no  model  and  are  fuUy  incremental.  The  PI  has  extensive 
experience  in  expressing  a  policy  based  on  dynamic  programming  [4,  13, 17].  However,  all  3  methods  are 
applicable  to  our  research  agenda.  For  example,  Monte  Carlo  techniques  can  be  used  in  one  incremental 
step  (or  one  episode)  if  a  sample  transition  probabilities  (not  a  complete  one)  exists.  Finally,  temporal 
differencing  is  the  combination  of  Monte  Carlo  and  dynamic  programming,  i.e.,  learning  directly  from 
raw  experience  without  the  model,  and  then  updating  estimates  based  in  part  on  learned  estimates. 

5  Experimental  Design 

Experiments  will  be  performed  at  the  Life  Sciences  Microscope  Resource  (LSMR,  http://mhbhl.lbl.gov/lsmr) 
using  our  VSOM  (Figures  6a-g).  This  Zeiss  Axiovert  135  H/DIC  inverted  microscope  is  used  for  trans¬ 
mitted  hght  (phase  and  DIC)  and  multicolor  fluorescence  microscopy.  It  is  equipped  with  a  computer 
controlled  xy  scanning  stage  (Figure  6b),  z-<ixis  stepping  motor  (Figure  6f),  and  a  six  position  filter 
wheel  (Figure  6d).  A  12-bit  XiHix  CCD  camera  (Xillix  Technologies,  Vancouver,  BC)  containing  a 
Kodak  KAF-1400  CCD  chip  (1317x1035  pixels,  7x7  micron  pixel  size)  wiU  be  used  for  these  studies 
(Figure  6a).  This  camera  has  a  readout  rate  of  8  MHz  (approximately  four  fuU  size  images  per  sec). 

The  intensified  video  camera  (Model  CCU-84,  Pulnix  America,  Inc.  and  robotic  arm  (Model  MP-185 
Motorized  Huxley- Wall  Type  Micromanipulator  from  Sutter  Instrument  Co.,  Novato,  CA)  used  for  our 
automated  DNA  microdissection  work  [11]  can  be  seen  in  Figures  6e  and  6g,  (respectively). 

For  the  work  proposed  here,  we  wiU  use  a  recently  acquired  (Peltier)  temperature  controlled  mi¬ 
croperfusion  chamber  (PDMI-2  open  chamber  with  TC-202  Bipolar  temperature  controller.  Harvard 
Apparatus/Medical  Systems  Research  Products,  HoUiston,  MA)  (See  Figure  7,  bottom  right).  We 
have  also  recently  acquired  one  computer-controlled  dual-syringe  pump  (Pump-33,  Harvard  Appara¬ 
tus/Medical  Systems  Research  Products,  HoUiston,  MA)  (See  Figure  7,  top  left).  The  inverted  design 
of  our  microscope  allows  perfusion  of  Uving  ceUs,  with  immobiUzed  ceUs  imaged  from  below.  An  oil- 
immersion  objective  can  be  seen  under  the  microscope  coversUp  in  Figure  6h.  Cells  may  be  grown 
in  35mm  Petri  dishes  or  grown  on  circular  covershps  and  viewed  from  below.  A  ceU  chamber  with  a 
circular  coversUp  base  is  seen  disassembled  in  Figure  7  (bottom  left).  With  this  inverted  microscope 
design,  it  is  possible  to  switch  from  fluorescence  to  transmitted  light  detection  (phase  or  DIC)  and 
verify  ceU  locations  and  morphology.  This  is  achieved  with  a  computer  controUed  transmitted  Ught 
shutter  (Figure  6c).  A  computer  controUed  epifluorescence  shutter  is  also  available  in  the  filter  wheel 
(Figure  6d).  VSOM  operation  of  the  system  (control  of  shutters,  filter  wheel,  xy  stage,  z-axis  motor, 
XiUix  camera)  will  be  soon  be  automated  via  a  software  control  panel  (i.e.,  a  Java-based  graphical  user 
interface  or  GUI). 

5.1  Mitochondrial  Membrane  Potential  and  Rhl23 

It  has  recently  been  recognized  that  mitochondria  play  a  central  role  in  the  regulation  of  programmed 
cell  death,  or  apoptosis  [7].  Changes  in  the  mitochondrial  membrane  potential  occur  during  the  early 
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stages  of  apoptosis.  Mitochondrial  defects  also  occur  in  cancer  [25].  Most  carcinoma-derived  cell  lines 
accumulate  and  retain  this  dye  much  more  than  do  normal  epithelial  cells  [21],  and  this  includes  the 
human  breast  cancer  cell  line  MCF-7  [5].  The  primary  factor  for  selective  accumulation  and  retention 
is  the  higher  mitochondrial  membrane  potential  in  these  cancer  cell  hnes  [5].  ^ 

There  are  several  advantages  in  choosing  Rhl23  for  initial  VSOM  studies.  Rhl23  Is  known  to  locahze 
in  the  mitochondria  of  cells,  the  mechanism  of  uptake  is  well  understood,  and  many  agents  are  available 
which  modulate  its  uptake  and  retention  [27].  In  addition,  the  ability  to  take  up  Rhl23  is  a  rehable 
indicator  of  cell  health  and  viability  [2],  it  is  highly  fluorescent,  and  its  uptake  can  be  monitored  at 
the  single  cell  level  using  fluorescence  microscopy.  It  has  often  been  used  as  a  marker  for  multi-drug 
resistance  (MDR). 

Kinetic  studies  with  other  rhodamine  dyes  such  as  tetramethyl  rhodamine  ethyl  ester  (TMRE) 
have  demonstrated  that  Rhl23  has  significant,  non-Nerstian  retention  properties,  because  it  is  not 
as  easily  washed  out  of  mitochondria  as  TMRE  [6].  Additionally,  the  negatively  charged  compound 
TPB  (tetraphenylborate)  was  shown  to  accelerate  cationic  dye  permeation,  probably  by  a  complicated 
mechanism.  We  will  include  TPB  exposures  in  some  of  our  VSOM  assays.  It  is  likely  that  the  non- 
Nerstian  components  of  Rhl23  retentions  will  contain  a  great  deal  of  cell-type  specific  information. 

5.2  Dynamics  of  MF  VSOM  experiments 

Real-time  experiments  wiU  be  performed  on  separate,  physically  separated  populations  (multiple  fields) 
of  cells.  Cells  will  be  grown  on  microscope  coverslips  or  in  35mm  Petri  dishes.  A  schematic  of  a  multi- 
field  VSOM  experiment  is  presented  in  Figure  8.  Several  images  will  be  acquired  for  one  cell  type  on 
one  half  of  the  coverslip  (gray  boxes  labeled  1  and  2  in  Figure  8),  and  then  the  xy  stage  will  bring  the 
other  half  of  the  coverslip  over  the  microscope  objective  for  imaging  the  second  cell  type  (gray  boxes 
3  and  4).  The  system  will  then  return  to  the  same  fields  of  view  on  the  opposite  side  of  the  coverslip, 
detect  the  same  cells,  and  obtair^  the  new  fluorescence  intensity  for  each  cell.  This  side  to  side  and 
field  to  field  motion  will  continue  throughout  the  experiment.  At  several  locations  on  the  coverslip, 
and  at  each  time  point,  both  a  transmitted  light  image  and  a  fluorescence  image  will  be  acquired.  The 
boundary  of  each  cell  will  be  determined  from  the  transmitted  light  image  (phase  contrast),  and  the 
amount  of  Rhl23  in  each  cell  will  be  determined  from  the  fluorescence  image.  Both  transmitted  and 
epifluorescence  shutters  will  remain  closed  except  when  images  are  being  acquired.  There  will  be  no 
physical  barrier  between  the  two  sides  of  the  coverslip.  Autofocusing  will  be  performed.  An  example  of 
Rhl23  stained  immortalized  human  mammary  epithelial  cells  (A1L5BR)  is  shown  in  Figure  9. 

Four  or  more  computer-controUed  syringe  pumps  will  introduce  Rhl23  and/or  other  compounds 
into  the  cell  chamber  (Figure  8).  By  controlling  the  rates  of  perfusion  of  each  syringe  separately,  a  wide 
range  of  concentrations  can  be  introduced  into  the  cell  chamber.  For  example,  syringe  A  could  contain 
a  very  concentrated  Rhl23  solution  that  can  be  diluted  with  buffer  from  syringe  B  (Figure  8).  Buffer 
only  from  syringe  B  could  be  delivered  for  wash-out  cycles. 

Figure  10  and  Table  11  (Figure  11)  illustrate  the  type  of  time-dependent  response  curves  that  are 
expected  from  typical  VSOM  baseline  experiments.  The  assay  described  in  Figure  11  and  illustrated 
schematically  in  Figure  10  was  constructed  by  referring  to  pubhshed  reports  that  compare  the  uptake 
and  retention  of  Rhl23  in  ’’normal”  epithehal  cells  (CV-1)  relative  to  ’’typical”  human  carcinoma 
epithelium  cells  (MCF-7)  [5].  The  responses  reported  in  these  comparative  studies  were:  Rhr23  uptake 
and  retention  as  a  function  of  high  potassium  (137mM  KCl,  3.6  mM  NaCl),  low  potassium  (137  mM 
NaCl,  3.6  mM  KCl),  temperature,  and  valinomycin  exposure.  Responses  due  to  nigericin/ouabain 


Annual  Report:  Callahan,  Parvin  Page  109 


were  also  studied.  Several  striking  differences  were  observed  between  normal  and  carcinoma  cells. 
Most  of  the  differences  observed  were  attributed  largely  to  differences  in  mitochondrial  membrane 
potential.  Although  we  do  not  expect  to  see  exactly  the  same  results  as  these  authors  reported,  VSOM 
aasays  that  target  the  mitochondrial  are  hkely  to  reveal  differences  in  the  mitochojpidria  physiology  of 
our  model  cell  systems.  We  will  use  KBMF  VSOM  assays  to  identify  and  optimize  specific  aspects 
of  mitochondrial  physiology  that  discriminate  between  cell  types  and  predict  doxorubicin  sensitivity. 
Successful  demonstration  of  these  preUminary  KBMF-VSOM  assays  will  give  us  valuable  insights  into 
the  strengths  and  weaknesses  of  this  technology  and  will  aid  the  future  development  of  more  complex 
KBMF-VSOM  assays 

5.3  Model  Cell  Systems 

MCF-7  Model  System:  The  MCF-7  (DS)  and  MCF-7adr  (MDR)  ceU  lines  are  an  ideal  model  system 
for  VSOM  system  development.  In  this  model  system,  two  fluorescence  assays  for  drug  response/multi¬ 
drug  resistance  are  already  available.  These  two  assays  are  complementary  and  allow  one  to  identify 
ceU  types  based  the  retention  of  fluorescence  dyes  (calcein-AM  and  Lysotracker-Blue,  Green,  or  Red) 
during  perfusion  experiments.  In  this  model  system,  ceUs  that  retain  calcein  are  drug-sensitive  (DS) 
and  lack  the  ability  to  pump  out  calcein.  This  retention  of  calcein  (green  fluorescence)  is  a  good  marker 
for  doxorubicin  cytotoxicity.  On  the  other  hand,  the  MDR  ceUs  in  this  model  system  possess  acidified 
vesicles  and  wiU  retain  Lysotracker-Red  (LT-R,  red  fluorescence).  It  has  been  shown  that  overcoming  a 
MDR  (MCF-7adr)  ceU’s  abiUty  to  pump  out  calcein  or  overcoming  an  MDR  (MCF-7adr)  ceU’s  ability 
to  sequester  LT-R  in  acidified  vesicles  (lysosomes)  are  both  exceUent  surrogate  markers  for  doxorubicin 
sensitization  and  demonstrate  the  potential  of  VSOM  fluorescence  assays.  We  wiU  test  these  cells  using 
the  Rhl23  assay  and  attempt  to  make  correlations  across  channels. 

HMEC  Model  CeU  System:  The  HMEC  hues  have  been  developed  here  at  LBNL  by  Dr.  Martha 
Stampfer  and  have  been  extensively  characterized  by  our  coUaborator,  cancer  biologist  Dr.  Paul  Yaswen 
(Stampfer  and  Yaswen,  1994).  We  wiU  develop  VSOM  fluorescence  assays  that  distinguish  between 
normal  finite  Ufespan,  immortalized,  and  tumorigenicaUy  transformed  human  mammary  epithelial  cells 
(HMECs)  (Stampfer  and  Yaswen,  1994).  In  addition  to  screening  these  three  ceU  Unes  with  currently 
available  assays  (calcein- AM/ verapamil  and  lysotracker/monesin),  we  wiU  perform  Rhl23  assays  in  a 
separate  channel  and  try  to  make  correlations  across  channels. 

6  Milestones 

Year  1:  Complete  integration  of  aU  system  components  that  include  (i)  a  specifically  designed  GUI,  (ii) 
on-Une  image  analysis  software,  (iii)  database  integration  for  offline  analysis,  (iv)  effective  visuaUzation 
of  ceU  responses  in  the  model  system,  and  (v)  development  of  offline  and  onUne  learning  techniques. 
Demonstrate  this  system  to  other  researchers  and  DOE  program  managers  remotely  using  the  architec¬ 
ture  developed  as  a  part  of  DOE2000  program. 

Year  2:  Complete  integration  and  testing  of  offline  and  online  learning  algorithms.  CoUect  data,  pop¬ 
ulate  the  database  with  baseline  assay  schedules.  Refine  the  database  by  altering  stimuli  and  environ¬ 
mental  conditions.  Demonstrate  successful  correlations  between  observed  mitochondrial  physiological 
responses  and  doxorubicin  sensitivity. 
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7  Budget  Explanation 

The  proposed  budget  includes  3  units  of  computer  controlled  syringes  to  be  acquired  (currently,  one 
unit  has  been  purchased).  i 

B.  Parvin  (0.25  FTE)  will  be  the  project  PI,  where  he  will  have  overall  project  responsibility.  He  will  | 
develop,  integrate,  and  test  proposed  learning  algorithms  for  the  system.  He  will  review  all  design  andi 
implementation  of  imaging  and  software  component  as  they  will  be  integrated.  He  wiU  be  responsible 
for  demonstration  of  the  proposed  system.  He  wiU  also  coordinate  publications  of  research  papers  for 
the  project. 

G.  Cong  (0.25  FTE)  wiU  be  responsible  for  integration,  testing  of  the  imaging  and  database  com¬ 
ponents.  He  wiU  implement  and  modify  the  GUI  is  needed. 

D.  CaUahan  (0.4FTE)  is  the  project  Co-PI,  and  wiU  be  responsible  for  the  design,  implementation, 
and  execution  of  VSOM  fluorescence  assays.  He  wiU  be  responsible  for  protocols  involving  ceU  biology 
and  ceU  culture.  He  wiU  specify  the  GUI,  and  populate  the  database  with  ceU  responses.  He  will 
execute  the  real-time  experiments  that  refine  the  VSOM  assays  in  the  database,  and  will  also  conduct 
the  experiments  that  search  for  differences  between  physically  separated  ceU  populations.  He  wiU  focus 
on  the  biomedical  aspects  of  the  project  and  attempt  to  rapidly  obtain  important  biological  results. 
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Figure  1:  Interaction  of  various  components  in  the  knowledge  based  VSOM.  The  itallic  letters  refer  to 
services  defined  in  the  overall  system  architecture  as  shown  in  Figure  2. 


Figure  2:  Interaction  Between  OMG  defined  Enabling  Services  and  funded  Services  through  the 
DOE2000  program. 
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Figure  3:  This  example  demonstrates  that  segmentation  of  nuclei  with  purely  global  or  local  operations 
is  not  feasible:  (a)  original  image;  (b)  threshold  image;  (c)  boundary  of  nuclei;  and  (d)  local  troughs. 


,(a-)  (b)  (c) 

Figure  4;  Segmentation  results;  (a)  original  image;  (b)  manual  segmentation;  (c)  our  results  on  seg¬ 
mentation  of  nuclei. 


(a)  (b)  (c) 

Figure  5;  Multiple  Channel  data  fusion  for  segmentation  of  nuclei:  (a)  DAPI  image  (blue  channel);  (b) 
Texas  Red  image  (red  channel);  (c)  segmentation  results  on  FITC  (green  channel). 
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Figure  8:  Schematic  of  multi-field  VSOM 


lOX,  fluoresconcc  only  lOX,  phase omtmst  only  lOX,  fluorescence  +  phase  contrast 


Figure  9;  Cell  images  acquired  under  different  imaging  modalities 
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VSOM  ASSAY  BASELINE  DATA  EXAMPLE 
5  Cell  Responses  shown  (out  of  hundreds  possible) 
(Rh123,  Vallnomycin,  High  and  Low  K+ 


Figure  10:  Cell  responses  containing  information  on  mitochondrial  physiology. 
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Phase  I :  VSOM  ASSAYS  FOR  BUILDING  UP  BASEL.INE  CELL  RESPONSES  IN  DATABASE 
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Abstract 


We  propose  to  image  inRNA  transcripts  in  real  time  in  tissue  culture  and  help  develop  antisense  compounds,  I 
delivery  systems,  and  signal  amplification  methods  that  will  lead  to  radiolabeled  antisense  gpmpounds  suit-  | 
able  for  real  time  medical  imaging.  Studies  will  be  performed  in  parallel  with  our  colleagues  at  the  LBNL  ‘ 
Center  for  Functional  Imaging  (CFI),  who  have  submitted  a  companion  grant  entitled,  “Strategies  for  Imag¬ 
ing  Gene  Expression  with  PET.”  In  both  proposals  we  will  use  the  same  three  oligonucleotide  sequences. 
We  will  target  the  same  mRNA  species  coding  for  the  bcl-2  protein,  and  we  will  use  the  same  human  breast 
cancer  cell  line,  BT-474,  as  a  model  system.  We  propose  to  track  fluorescently  labeled  antisense  compounds 
within  individual  living  cells,  guide  them  to  the  appropriate  target  mRNA  in  the  cell,  and  determine  whether 
individual  cells  are  affected  in  the  manner  predicted.  Using  digital  imaging  fluorescence  microscopy,  novel 
computational  techniques,  and  bioinformatic  tools,  we  will  track  the  fluorescent  antisense  compound  in  one 
channel,  while  monitoring  other  cell  reponses,  such  as  apoptosis,  in  separate  channels.  We  propose  to  image 
the  expression  of  bcl-2  at  the  mRNA  level  as  a  function  of  time,  external  cell  stress,  and  antisense  compound 
structure.  We  will  correlate  these  observations  with  bcl-2  protein  levels  in  each  cell  and  the  ultimate  fate 
of  each  cell.  We  believe  such  experiments  will  provide  detailed  information  on  the  biophysical  properties 
of  specific  antisense  compounds.  This  type  of  data  acquisition  will  aid  the  intelligent  design  of  antisense 
compounds  useful  for  imaging  gene  expression  with  PET. 
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1  Specific  aims 

1.  Aid  the  intelligent  design  of  antisense  compounds  useful  for  imaging  specific  mRNA  targets 

in  living  cells.  ^ 

2.  Collect  and  store  detailed  information  on  the  biophysical  properties  of  specific  antisense  com¬ 
pounds  as  it  is  guided  to  specific  mRNA  targets  in  the  cytoplasm  of  living  cells. 

3.  Construct  an  imaging  system  for  automatic  measurement  of  dynamic  time  varying  expression 
of  mRNA  as  a  function  of  a  particular  antisense  compound. 

4.  Develop  and  implement  a  policy  for  stepwise  refinement  and  design  of  antisense  compounds 
based  on  previously  measured  biophysical  properties  from  the  imaging  system. 

2  Narrative 

The  full  potential  of  antisense  oligodeoxyribonucleotides  (AS-ODNs)  has  yet  to  be  realized  in  living 
cells  (in  vitro)  or  in  in  living  tissue  (in  vivo).  Successful  demonstration  of  the  use  of  fluorescently 
labeled  AS-  ODNs  as  real-time  gene  expression  imaging  agents  in  living  cells  in  vitro  will  assist 
efforts  to  use  AS-ODNs  as  selective  agents  for  the  perturbation  of  gene  expression.  In  addition, 
data  provided  by  our  proposed  experiments  will  be  extremely  valuable  for  the  design  and  ongoing 
modifications  of  these  agents  by  radiochemists,  who  seek  to  use  these  compounds  as  medical  imag¬ 
ing  agents  in  vivo.  In  our  proposed  fluorescence  microscope  studies  on  living  cells,  correlations  will 
be  made  between  specific  chemistries  and  the  observed  behaviors  of  fluorescently  labeled  AS-ODNs. 
Often,  fluorescently  labeled  antisense  compounds  are  sequestered  within  non-cytoplasmic  compart¬ 
ments  of  the  cell  (Leonetti  et  ah,  1991  [20];  Thaden  and  Miller,  1993  [37]).  By  determining  ways 
to  release  AS-ODNs  from  sequestration  in  intracellular  compartments  and  induce  redistributions 
within  individual  cells,  we  will  learn  a  great  deal  about  that  compound’s  biophysical  properties  in 
the  cell.  The  acquisition  of  this  type  of  data  will  aid  the  intelligent  design  of  AS-ODNs  useful  for 
imaging  specific  mRNA  targets  in  living  cells.  Specifically,  our  experiments  will  help  radiochemists 
decide  which  AS-  ODN  modifications  and  delivery  methods  target  mRNA  in  the  cytoplasm  most 
effectively. 

We  believe  we  have  important  new  technology  and  the  interdisciplinary  team  necessary  to  make 
progress  on  these  difficult  technical  problems.  We  propose  to  use  visual  servoing  optical  microscopy 
(VSOM)  to  automatically  “guide”  specific  AS-ODNs  to  specific  mRNA  targets  in  the  cytoplasm  of 
living  cells  and  observe  the  subsequent  cellular  response  on  a  cell-by-cell  basis.  Visual  servoing  (a 
term  from  the  field  of  robotic  vision  research)  refers  to  to  dynamic  control  of  experimental  param¬ 
eters  based  on  automatic  interpretation  of  the  information  contained  in  a  stream  of  digital  images. 
Our  novel  image  processing  algorithms  give  us  the  ability  to  automatically  detect  the  amount  of  flu¬ 
orescently  labeled  AS-ODN  in  the  individual  compartments  of  individual  living  cells  in  real  time. 
Advances  in  the  selective  fluorescence  staining  of  living  cells,  including  the  various  intracellular 
compartments  in  living  cells,  together  with  advances  in  multicolor,  digital  imaging  fluorescence 
microscopy  will  allow  us  to  identify,  in  real  time,  the  subcellular  compartment(s)  where  a  given 
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fluorescently  labeled  AS-ODN  is  located.  We  currently  have  access  to  a  large  selection  of  biological 
and  chemical  agents  that  can  perturb  these  compartments  specifically  and,  for  example,  collapse 
the  compartmental  membrane  potentials  that  are  required  for  proper  function.  Using  VSOM  and 
a  set  of  computer-controlled  perfusion  pumps,  we  will  perturb  living  cells  in  very  specific  ways  as 
we  continuously  monitor  physiological  responses  and  correlate  these  responses  with  the  movement 
of  fluorescently  labeled  AS-ODNs  between  different  compartments  of  the  cell.  Our  experiments 
will  provide  details  on  the  biophysical  properties  of  each  AS-ODN  as  it  moves  into  and  within  the 
cell  in  response  to  computer-applied  stimuli.  We  try  to  explain,  using  biophysical  principles,  why 
the  compounds  partition  into  various  compartments,  and  predict  how  the  compounds  should  be 
altered  to  prevent  such  problems  in  the  future. 

As  documented  in  the  extensive  in  vitro  literature  on  this  subject,  fluorescently  labeled  AS- 
ODNs  rarely  enter  the  cell  via  simple  diffusion  from  solution  phase  through  the  plasma  membrane 
to  an  inRNA  target  in  the  cytoplasm  (Shoji  et  al.,  1991  [35])  (Thierry  and  Dritschilo,  1992  [38]) 
(Thaden  and  Miller,  1993  [37]).  In  fact,  even  when  AS-ODNs  are  microinjected  into  the  cytoplasm, 
they  can  rapidly  migrate  to  the  nucleoplasmic  (nuclear)  compartment.  This  precludes  their  use  as  a 
means  of  quantifying  copy  number  of  specific  mRNA  molecules  in  the  cytoplasm.  Examples  of  this 
behavior  have  been  reported  for  phosphodiester  (PD),  phosphorothioate  (PT),  and  methylphospho- 
nate  (MP)  forms  of  the  same  fluorescently  tagged  28-mer  (Chin  et  al.,  1990  [10]).  This  and  several 
other  technical  difficulties  are  associated  with  the  use  of  fluorescently  labeled  AS-ODNs  for  the 
detection  or  modification  of  gene  expression  .  True  antisense  activity  requires  nuclease  resistance, 
cellular  uptake,  and  hybridization  to  a  specific  target  by  Watson-Crick  base  pairing.  Other,  non¬ 
antisense  (non-sequence-specific)  mechanisms  of  action  by  AS-ODNs  are  often  observed  (Yaswen 
et  al.,  1993  [43]).  However,  the  exquisite  sequence  specificity  of  Watson-Crick  base  pairing  and  the 
tremendous  sensitivity  of  fluorescence  signal  detection  motivate  us  to  overcome  these  problems. 

One  problem  that  must  be  overcome  is  the  sequestration  of  AS-ODN  in  lysosomes.  Lysosomes 
are  acidic  compartments  of  the  cell  containing  a  wide  range  of  enzymes  for  degradation  of  unwanted 
or  toxic  cellular  materials.  Using  VSOM  and  computer  controlled  perfusion  pumps,  we  propose  to 
track  fluorescently  labeled  AS-ODNs  within  individual  living  cells,  automatically  apply  the  proper 
agent(s)  at  the  proper  concentration  for  the  proper  length  of  time,  and  in  this  way  release  the 
fluorescently  labeled  AS-ODN  from  cellular  lysososomes.  To  assist  us  in  this  effort,  we  will  develop 
a  bioinformatics  database  that  will  record  the  specific  perturbations  applied  and  the  resulting 
effect  on  the  intracellular  redistribution  of  fluorescently  labeled  AS-ODNs  within  the  cells.  Such  a 
database  will  become  extremely  valuable  for  the  design  and  ongoing  modifications  of  these  agents 
by  radiochemists,  especially  as  correlations  are  made  between  specific  chemistries  and  the  observed 
behaviors  of  the  AS-ODNs.  We  believe  the  ability  to  perform  data  mining  and  visualization  of 
multidimensional  data  representing  actual  intracellular  behavior  of  these  compounds  will  allow  us 
to  rapidly  identify  critical  biological  determinants  of  compound  localization,  specificity,  and  toxicity. 

Other  problems  to  be  overcome  are  (a)  the  tendency  of  AS-ODNs  to  bind  in  a  nonspecific 
manner  to  proteins  in  the  nucleus  and  accumulate  there  instead  of  binding  to  mRNA  targets  in  the 
cytoplasm,  and  (b)  failure  of  some  AS-ODNs  to  elicit  the  desired  response  on  gene  expression  after 
they  successfully  reach  the  cytoplasm.  Thus,  in  addition  to  using  VSOM  to  guide  fluorescently 
labeled  .4S-ODNs  to  the  appropriate  target  mRNA  sequence  in  the  cytoplasmic  compartment  of 
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the  cell,  we  will  also  use  it  to  determine  whether  individual  cells  are  perturbed  in  the  manner 
predicted  by  the  mRNA  sequence  targeted.  Using  multicolor  (or,  “multichannel”)  digital  imaging 
fluorescence  microscopy,  we  will  monitor  the  fluorescent  AS-ODN  in  one  color,  or  “channel”,  while 
we  simultaneously  monitor  other  physiological  cell  responses  in  separate  channels.  this  way,  we 
hope  to  successfully  image  expression  (and  alterations  in  expression)  of  specific  genes  at  the  mRNA 
level  as  a  function  of  time,  external  cell  stimulation,  and  AS-ODN  chemical  structure.  Specifically, 
we  will  verify  alterations  in  gene  expression  by  quantifying  alterations  in  bcl-2  protein  levels  in  two 
different  ways  (i)  by  observing  alterations  in  apoptosis  in  stressed  cells  (ii)  by  returning  to  the  same 
field  of  cells  after  VSOM  experiments  and  observing  levels  of  bcl-  2  protein  on  a  cell-by-cell  basis 
using  indirect  immunofluorescence. 


3  Background  and  significance 

3.1  Fluorescence  microscope  sensitivity  and  resolution 

Recently,  it  has  been  demonstrated  that  a  single  fluorescent  molecule  can  be  detected  in  aqueous 
solution  at  room  temperature  (Nie  et  ah,  1994).  This  was  done  in  real  time,  without  statistical 
analysis  on  a  moving,  single- chromophore  molecule  (fluorescein).  Single  DNA  bases  labeled  with 
one  fluorescent  tag  were  also  observed.  To  achieve  this,  a  laser  confocal  microscope  with  a  laser 
probe  volume  (or  resolution  element)  of  0.5  femtoliters  (S.OAlO'^^/jters)  was  used  with  10  /.iL 
of  3.3X10“^®  M  fluorescein  sandwiched  between  two  0.13  mm  thick  microscope  coverslips.  The 
excitation  wavelength  was  488nm.  A  photon-counting  Si  avalanche  photodiode  was  used  as  a 
detector,  with  a  high  numerical  aperture  (NA  1.3)  microscope  objective  lens,  and  as  few  optical 
elements  as  possible  in  the  optical  path.  The  pH  of  the  solution  was  12,  and  contained  lOOmM 
mercaptoethanol  to  prevent  photobleaching  by  scavenging  oxidative  species.  Fluorescein  is  a  pH 
sensitive  molecule  which  is  very  photosta.ble  with  a  high  quantum  yield  under  these  conditions. 
Although  it  was  not  noted  by  the  authors,  coverslips  are  often  chemically  treated  before  these 
types  of  experiments  in  order  to  remove  fluorescent  impurities.  The  authors  did  note  their  ability 
to  detect  low  levels  of  fluorescent  impurities  in  the  aqueous  solution. 

In  this  proposal,  we  will  not  attempt  to  detect  single  molecules  of  AS-ODNs  labeled  with  a 
single  fluorescent  tag  for  the  following  reasons.  Living  cells  have  a  certain  inherent  level  of  autoflu¬ 
orescence  that  can  obscure  the  fluorescence  signal  from  single  molecules  of  antisense  compounds 
labeled  with  a  single  fluorescent  probe.  In  addition,  it  is  not  appropriate  to  use  intense  laser  illumi¬ 
nation  or  anti-photobleaching  agents  when  studying  living  cells.  The  autofluorescence  of  one  entire 
cultured  NIH  3T3  fibroblast  excited  at  488nm  has  been  estimated  to  be  equivalent  to  approximately 
34,000  fluorescein  molecules,  on  average.  The  major  sources  of  autofluorescence  in  mammalian  cells 
are  the  flavin  coenzymes  (FAD  and  FMN,  absorption/emission  450/515nm)  and  reduced  pyridine 
nucleotides  (NADH,  absorption/emission  340/460nm).  Other  biomolecules  are  also  fluorescent,  in¬ 
cluding  nicotinamide,  adenine  nucleotides,  riboflavin,  porphyrins,  and  aromatic  amino  acids  such 
as  tryptophan.  One  can  adopt  strategies  to  minimize  the  contributions  of  this  intracellular  back¬ 
ground  fluorescence.  For  example,  NADH  is  concentrated  in  cellular  mitochondria;  thus,  one  can 
avoid  making  fluorescence  measurements  in  areas  of  the  cell  occupied  by  a  mitochondrion.  This  is 
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not  difficult  when  performing  measurements  on  living  cells  (grown  on  microscope  coverslips)  using 
a  high  NA,  oil-  immersion  microscope  objective,  where  the  2D  diffraction-limited  resolution  interval 
is  approximately  0.2/t.  The  mitochondrial  “compartment”  is  one  cellular  compartment  we  will  be 
monitoring  in  our  experiments. 

3.2  Previous  collaborations  and  results 

Our  group  has  demonstrated  our  experience  with  the  detection,  manipulation  and  microdissection 
of  single  DNA  molecules  (Parvin  et  ah,  1997)  (included  in  appendix)  [30].  As  discussed  above, 
special  experimental  conditions  must  be  maintained  during  these  experiments.  For  example,  in  our 
DNA  microdissection  experiments,  DNA  molecules  were  stretched  and  oriented  by  flow,  then  fixed 
in  place  on  a  coverslip  that  had  been  specially  cleaned  to  decrease  fluorescence  background.  This 
allowed  us  to  search  for  stationary  fluorescence  signal  in  a  specific  focal  plane  (near  the  coverslip), 
and  no  fluorescence  signal  existed  above  or  below  this  focal  plane  that  could  obscure  the  signal. 
In  addition,  the  signal  pattern  was  distinct:  linear  rods,  all  of  equivalent  length,  oriented  in  the 
same  direction.  Before  illumination,  the  molecules  were  layered  with  a  glycerol  solution  containing 
a  fluorescent  dye  and  an  anti-  photobleaching  compound.  The  fluorescent  dye  molecule  used  for 
staining  (YOYO-1,  Molecular  Probes,  Eugene,  OR)  bound  once  every  2-4  base-pairs,  so  that  many 
fluorochromes  were  concentrated  in  a  single  pixel  of  the  digital  image.  In  addition,  the  fluorescence 
of  this  dye  is  greatly  enhanced  when  it  binds  to  DNA,  so  that  unbound  molecules  contribute  little 
to  the  fluorescence  background.  In  mRNA  fluorescence  microscope  studies,  “tracks”  of  mRNA 
have  been  detected  suggestive  of  a  transport  route  from  the  site  of  transcription  to  the  nuclear 
membrane  after  being  induction  of  the  gene  by  extracellular  stimulation  (Raap  et  ah,  1991)  [33]. 
Others  have  reported  specific,  sensitive,  and  quantitative  detection  of  mRNA  for  the  measurement 
of  gene  expression  in  individual  cells  using  fluorescence  microscopy,  in  situ  hybridization  protocols 
and  highly  fluorochrome  labeled  gene  probes  (Pachmann  et  ah,  1991  [25]).  Using  a  microscope 
photometer,  it  was  possible  to  detect  between  1  and  20  fluorochrome  labeled  gene  probes  above 
the  autofluorescence  of  fixed  cells.  In  addition,  RNA  tracks  and  RNA  splicing  have  been  observed 
in  mammalian  interphase  nuclei  after  induction,  with  unspliced  transcripts  accumulating  in  one  or 
two  discrete  sites  in  the  nucleus. 

We  have  demonstrated  that  single  copies  of  viral  genomes  inserted  into  specific  sites  on  human 
chromosomes  can  be  detected  and  quantified  in  metaphase  chromosome  spreads  (Callahan  et  ah, 
1992  [8])  (included  in  appendix).  In  addition,  we  have  a  great  deal  of  experience  wdth  quantitative 
fluorescence  microscopy  (Callahan  et  ah,  1999  [6])  (included  in  appendix),  and  time-lapse  videomi¬ 
croscopy  (Barcellos-Hoff,  1992  [4])  of  living  cells,  3D  fluorescence  microscope  studies  (Lesko  et  ah, 
1995  [21])  and  the  use  of  fluorescence  standards  (Lesko  et  ah,  1989  [22])  (Kaplan  and  Picciolo,  1989 
[19]).  We  have  also  published  circular  dichroism  (CD),  NMR,  and  fluorescence  spectroscopy  studies 
of  the  impressive  ability  of  ODNs  to  adopt  a  wide  range  of  conformations  (Callahan  and  Hooker, 
1987  [7];  Callahan  et  ah,  1991  [9];  Kan  et  ah,  1991  [18])  even  when  the  ODNs  are  extensively  altered, 
as  in  the  case  of  oligonucleotides  with  methylphosphonate  backbones.  However,  this  plasticity  of 
structure  comes  at  a  price,  because  modified  AS-ODNs  can  adopt  unforeseen  conformations  in 
solution,  and  bind  to  unforeseen  targets  (Yaswen  et  ah,  1993  [43]).  In  addition,  mRNA  targets 
themselves  can  adopt  conformations  that  make  the  complementary  target  sequence  inaccessible. 
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3.3  Sequestration  of  drugs  in  lysosomes 

Drug  delivery  problems  has  unpredicted  side  effects  and  are  not  unique  to  antisense  compounds. 
These  technical  difficulties  are  well  known  to  developers  of  new  drugs,  and  drug  delivery  systems. 
Only  the  first  generation  of  ODNs  are  currently  in  clinical  trials,  and  the  insights  gained  from  these 
trials  will  lead  to  chemical  modifications  that  will  improve  the  stability,  potency,  and  safety  of  AS- 
ODNs  (Temsamani  and  Guinot,  1997  [36]).  Analogous  problems  have  been  discussed  in  one  of  our 
recent  publications  where  the  amount,  distribution  pattern,  and  site  of  intracellular  sequestration 
were  identified  in  a  human  glioma  cell  line  for  the  fluorescent,  boronated  protoporphyrin  (BOPP) 
(CaUahan  et  al.,  1999  [6])  (included  in  appendix).  BOPP  is  currently  a  candidate  for  both  boron 
neutron  capture  therapy  (BNCT)  and  photodynamic  therapy  (PDT)  of  glioblastoma  multiforme 
(GBM).  We  were  able  to  identify  several  factors  that  influenced  the  uptake,  sequestration  and 
retention  of  BOPP  in  living  cells.  Using  lysosome  or  mitochondria-specific  fluorescent  probes,  we 
were  able  demonstrate  that  BOPP  accumulated  in  the  lysosomes  of  human  glioma  cells  in  vitro,  and 
not  in  the  mitochondria,  as  reported  for  C6  rat  glioma  cells.  We  were  also  able  to  determine  that 
BOPP  uptake  was  LDL-receptor  dependent,  and  was  also  dependent  on  the  amount  of  lipoproteins 
in  the  medium.  This  uptake  and  subcellular  trafficking  behavior  is  analogous  to  that  seen  for 
a  variety  of  fluorescently  labeled,  modified  AS-ODNs  that  demonstrate  a  vesicular  subcellular 
distribution  suggestive  of  an  endosomal/lysosomal  compartmentalization  after  endocytic  uptake 
(Gewirtz  et  ah,  1998  [17])  (Shoji  et  al.,  1991  [35])  (Thierry  and  Dritschilo,  1992  [38])  (Thaden  and 
MiUer,  1993  [37]). 

In  regards  to  BOPP,  it  should  be  noted  that  subcellular  localization  of  porphyrins  is  deter¬ 
mined  by  their  different  functional  groups.  Using  fluorescence  microscopy,  researchers  have  been 
able  develop  structure-localization  relationships  for  porphrins  (Woodburn  et  al.,  1991).  Porphyrins 
with  predominantly  cationic  side  chains  were  seen  to  localize  in  mitochondria,  while  those  with  a 
more  ionic  character  tended  to  localize  in  lysosomes.  A  third  class  of  compounds  showed  general 
cytoplasmic  staining.  It  is  of  interest  to  note  that  BOPP  appears  able  to  alter  its  site  of  intra¬ 
cellular  location  depending  on  the  cell  type  under  study  and  the  intracellular  environment  (i.e., 
rat  glioma  versus  human  glioma  cell  lines)  (Callahan  et  al.,  1999).  Structure-activity  studies  have 
been  performed  that  attempt  to  express  physicochemical  properties  of  such  compounds  numer¬ 
ically  (e.g.,  electric  charge,  pKa->  hydrophilicity-lipophilicity,  conjugated  bond  number,  etc)  and 
employ  fluorescence  microscopy  to  identify  the  favored  site  of  intracellular  localization  (i.e,  lyso¬ 
somes  with  low  internal  pH  or  mitochondria  with  high  internal  pH)  (Rashid  and  Horobin,  1990; 
Rashid  et  ah,  1991).  It  was  noted  that  inhibition  of  the  lysosomal  proton  pump  could  lead  to 
release  of  acidotropic  probes,  as  these  probes  diffuse  out  of  the  lysosome  as  its  internal  pH  rises. 
In  addition,  for  both  lysosomal  and  mitochondrial  specific  fluorescence  probes,  it  was  noted  that 
the  proper  balance  between  pKa,  charge,  lipophilicity  and  hydrophilicity  was  critical  for  the  out¬ 
come  of  the  probe-  cell  interaction.  This  is  reminiscent  of  current  attempts  to  optimize  .48-  ODN 
hydrophobicity  and  charge  by  creating  hybrid  ODNs  which  contain  backbones  composed  of  mix¬ 
tures  of  negatively  charged  phosphorothioate  linkages  and  nonpolar  methylphosphonate  linkages 
(Temsamani  and  Guinot,  1997  [36]). 

One  important  factor  of  visual  servoing  (to  be  followed  in  this  section)  will  be  the  exchange 
of  buffers  (and  compounds)  via  computer-controlled  perfusion  pumps.  These  various  extracellular 
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buffers  will  be  perfused  into  a  temperature  controlled  microperfusion  chamber,  and  the  responses 
of  hundreds  of  living  cells  will  be  monitored  and  analyzed  in  real  time. 

3.4  Previous  studies  in  fluorescently  labelled  AS-ADNs 

We  are  particularly  interested  in  previous  studies  that  have  demonstrated  the  differential  effects 
of  environmental  parameters  and  delivery  systems  on  the  intracellular  distribution  and  trafficking 
of  fluorescently  tagged  AS-ODNs  with  different  backbone  compositions.  For  example,  it  has  been 
demonstrated  using  fluorescently  tagged  and  labeled  PD  and  MP  (methylphosphonate)  AS- 
ODNs  that  changes  in  the  uptake  and  intracellular  trafficking  of  PD  and  PT  (phosphorothioate) 
ODNs  are  different  than  those  seen  for  MP  ODNs.  This  has  been  observed  by  varying  external 
parameters  such  as  temperature,  pH,  or  presence  of  excess  unlabeled  ODN  are  varied.  In  fluores¬ 
cence  microscope  studies  using  CH^C5  cells  (multidrug  resistant  Chinese  Hamster  Ovary  Cells), 
it  was  possible  to  inhibit  the  the  uptake  of  labeled  PD  ODNs  using  excess  unlabeled  PD 
ODN,  but  it  was  not  possible  to  inhibit  the  uptake  of  fluorescence  tagged  MP  ODN  using  excess 
unlabeled  PD  ODN.  In  addition,  when  the  external  pH  was  decreased  to  pH  5.5,  an  intracellular 
pH  sensitive  probe  (BCECF)  indicated  that  the  intracellular  pH  rapidly  (within  2  min)  decreased 
to  pH  5.7  (Shoji  et  ah,  1991  [35]).  Once  again,  a  difference  in  ODN  uptake  was  observed,  with 
uptake  of  PD-ODN  being  completely  inhibited,  while  uptake  of  MP-ODN  was  not  inhibited.  In 
both  cases,  MP-ODN  was  seen  to  localize  in  the  endosomal/lysosomal  compartment.  However, 
others  have  reported  that  when  MP-ODNs  are  directly  injected  into  the  cytoplasm  of  several  dif¬ 
ferent  cell  types,  they  are  rapidly  accumulated  in  the  nucleus,  as  are  PD  and  PT  ODNs  (Chin  et 
al.,  1990  [10];  Leonetti  et  al.,  1991).  Thus,  charge  effects  of  the  ODN  backbone  do  not  alter  the 
localization  of  microinjected  ODNs  into  the  cell  nucleus.  However,  the  3D  intranuclear  distribution 
observed  is  affected  by  the  chemistry  of  the  internucleoside  linkages.  For  example,  PD,  and  to  a 
lesser  extent,  PT  ODNs  colocalize  with  small  nuclear  ribonuclear  proteins  (snRNPs),  while  MP 
ODNs  colocalize  with  concentrated  regions  of  genomic  DNA  (Chin  et  al.,  1990).  The  nature  of  the 
fluochrome  used  for  labeling  does  not  alter  the  characteristics  of  nuclear  localization  (Leonetti  et 
al.,  1991).  More  detailed  studies  of  the  in  vitro  cellular  pharmacokinetics  of  three  types  of  ODNs 
(including  an  end-capped,  or  capPT  ODN)  have  been  performed  using  a  liposomal  delivery  system 
(Thierry  and  Dritschilo,  1992  [38]).  In  these  studies,  cellular  uptake,  intracellular  stability,  efflux 
rate  and  intracellular  localization  were  assessed  for  fluorescently  tagged  version  of  these  PD  and 
PT  ODNs  with  and  without  liposome  encapsulation.  Cellular  penetration  of  fluorescence  tagged 
PT  ODNs  was  greatly  facilitated  by  the  use  of  liposomes  created  using  the  method  of  minimum 
volume  entrapment  (MVE).  In  addition,  PD  ODNs  were  protected  from  degradation  (for  up  to  5 
days)  in  cell  culture  medium  when  encapsulated  in  liposomes. 

3.5  Visual  servoing 

Visual  servoing  aims  at  automated  manipulation  of  instrument  control  parameters  based  on  au¬ 
tomatic  image  interpretation.  A  simple  example  of  visual  servoing  is  autofocusing  for  an  optical 
microscope,  where  the  microscope  objective  is  moved  automatically  along  the  Z  direction  to  com¬ 
pensate  for  defocus.  In  this  context,  defocus  is  quantified,  and  the  Z-axis  is  moved  under  computer 
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control  for  optimum  focus.  A  more  complex  example  includes  drift  compensation  or  motion  detec¬ 
tion  as  a  function  of  external  stimuli  for  dynamic  experiments  [27].  This  system  (currently  in  use 
at  LBNL)  not  only  detects  motion  in  the  scene,  but  also  records  and  logs  pertinent  attributes(e.g. 
size,  shape,  response,  direction  of  motion).  This  information  is  then  compared  tg  attributes  of 
similar  events  that  are  stored  in  a  database.  Another  benefit  of  this  level  of  automation  is  that 
these  dynamic  experiments  can  be  conducted  remotely  over  the  wide  area  network  [30].  Automatic 
drift  compensation  provides  a  stationary  view  of  the  experiment  and  eliminates  the  need  for  high 
speed  network  and/or  quality  of  service. 

The  proposed  research  requires  close  collaboration  between  biophysicists,  chemists,  biologists, 
and  computer  scientists.  The  investigators,  listed  in  this  proposal,  have  a  proven  track  record  of 
scientific  collaboration  [26,  30,  31].  From  a  computer  science  perspective,  the  following  components 
are  needed  to  achieve  the  research  objectives:  (1)  computer  control  of  the  microscope  and  syringes; 
(2)  detailed  segmentation  of  cells  imaged  on  single  or  multiple  channels;  (3)  automated  annotation  of 
subcellular  responses  as  a  function  of  a  compound  and  its  various  attributes;  (4)  offline  correlation, 
clustering,  and  learning  techniques  for  constructing  a  model  of  how  various  compounds  target 
a  specific  compartments;  (5)  online  learning  techniques  for  dynamic  control  and  estimation  of  a 
policy  (schedule)  for  freeing  a  compound  from  a  site  and  guiding  it  (in  a  weak  sense)  to  a  specific 
compartment  in  the  cell.  The  interaction  between  various  system  components  are  shown  in  Figure 
1. 


3.6  Relation  to  other  projects 

The  computational  aspect  of  this  proposal  will  leverage  four  research  programs  that  are  funded 
through  DOE-MICS  {DOE2000  Material  and  MicroCharacterization  Collaboratory) ^  DOE-OBER 
{An  Imaging  Bioinformatic  Systeni  for  Studying  the  Effect  of  Low  Dose  Radiation),  a  DOD  breast 
cancer  research  grant  ( Visual  Servoing  for  Anti  Cancer  Drug  Uptake),  and  an  LDRD  grant  to  for 
feature  based  image  representation.  The  first  grant  provides  the  basis  for  remote  collaborative 
microscopy,  seamless  data  acquisition,  and  instrument  control.  The  second  grant  provides  the 
imaging  bioinformatic  component  for  annotating  data  and  images  and  the  needed  core  technology 
for  segmentation  of  cells.  The  third  grant  provides  a  platform  for  VSOM  that  will  be  available 
at  the  end  of  the  current  fysical  year.  The  fourth  grant  focuses  on  novel  algorithm  development 
for  motion  tracking  and  modeling.  Our  current  VSOM  grant  (funded  by  the  DOD  breast  cancer 
research)  does  not  include:  (1)  database  component  and  population  of  that  database  with  cell 
responses;  (2)  learning,  clustering,  and  data  mining  in  that  database;  (3)  real-time  analysis  and 
manipulation  of  physically  separated  population  of  cells;  (4)  detailed  segmentation  of  subcellular 
compartment  with  or  without  using  data  fusion  technique  (images  acquired  on  multiple  channels); 
(.5)  incorporation  of  large  number  of  syringes;  and  (6)  study  of  mitochondrial  physiologies  or  other 
compartments  in  the  cell.  We  are  confident  that  by  coupling  this  proposal  with  our  currently 
funded  programs,  a  new  biomedical  technology  can  be  demonstrated. 

Section  3  provides  an  overview  of  background  and  significance.  Section  4  provides  a  review  of 
core  technology  for  segmentation  of  cells.  Section  5  outlines  methods  and  deliverables. 
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Figure  1:  Interaction  of  various  components  in  the  knowledge  based  VSOM.  The  italic  letters  refer 
to  services  defined  in  the  distributed  architecture  (part  b  funded  by  DOE-MICS). 

4  Preliminary  studies 

The  keys  to  automated  analysis  of  mRNA  expression  are  subcellular  partitioning  of  cells  (seg¬ 
mentation),  on-line  dynamic  and  trajectory  analysis  of  antisense  compounds,  effective  storage  of 
computed  features,  and  large  scale  population  studies.  Our  group  has  developed  a  number  of  novel 
algorithms  to  enable  measurement  of  gene  expression  in  different  compartments  of  a  cell.  These 
techniques  are  constantly  being  refined  and  updated  with  new  features  and  capabilities.  In  sections 
4.2  and  4.3,  a  brief  overview  of  these  novel  algorithms  are  presented.  One  class  of  algorithms  focuses 
on  segmentation  of  images  derived  from  a  single  channel.  The  other  class  focuses  on  segmentation 
of  images  derived  from  multiple  channels. 

This  section  provides  a  summary  of  the  system  architecture  and  its  current  features  and  then 
outlines  the  details  of  segmentation  techniques. 

4.1  System  architecture 

The  current  functional  architecture  for  visual  servoing  is  shown  in  Figure  2.  It  consists  of  an 
image-acquisition  module,  image-analysis  module,  servoing,  and  archive.  The  image-acquisition 
module  provides  the  means  for  time-lapsed  high  resolution  video  microscopy  with  six  excitation 
filters.  This  module  has  a  recipe  manager  to  allow  either  manual  or  preprogrammed  capture  of 
images  at  different  temporal  and  excitation  frequencies.  The  analytical  module  has  a  pool  of 
unique  segmentation  algorithms  that  provide  a  feature-based  summary  of  images  based  on  an 
attributed  graph  model.  Accordingly,  each  node  in  the  graph  corresponds  to  a  homogeneous  region 
(e.g.,  nucleus,  cytoplasm)  in  the  image.  Attributes  of  each  node  include  the  bounding  contours, 
parametric  representation  of  this  contour  with  hyperquadrics,  a  number  of  derived  features,  and 
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the  response  of  subcellular  compartment  under  observation.  The  links  in  the  graph  encode  the 
adjacency  relationship  between  various  nodes.  The  attributed  graph  model  completely  expresses 
the  structural  definition  of  each  cell  and  its  neighborhood.  The  servoing  module  controls  the 
concentration  of  various  compounds  in  the  tissue  culture  vessel  by  controlling  the  floy  in  any  of  the 
four  syringes  that  sit  next  to  the  microscope.  The  servoing  provides  three  operational  modes  with 
each  mode  subsequently  registered  with  the  recipe-manager:  (1)  a  static  recipe  in  which  the  flow 
rate  of  each  compound,  its  start  point  in  time,  and  its  duration  is  specified;  (2)  a  dynamic  recipe 
in  which  the  flow  rate  and  its  duration  are  altered  as  a  function  of  a  particular  response  of  the  cells 
under  observation;  and  (3)  a  modulating  recipe  under  program  control  that  turns  the  flow  on  and 
olT  at  a  specifled  frequency.  The  archival  system  stores  the  images,  their  computed  graph  model, 
and  annotation  data  in  a  flat  file  environment.  One  major  feature  of  our  current  system  is  that  it 
can  be  operated  remotely  at  multiple  sites.  This  unique  feature  allows  researchers  to  collaborate 
on  a  given  experiment  while  sitting  at  geographically  remote  locations. 


Figure  2:  Functional  view  of  the  system  indicates  two  unique  features:  (1)  closed  loop  servo  control 
of  compounds  being  injected  into  petri  dish  based  on  detailed  analysis  of  images;  and  (2)  feature- 
based  storage  of  images  based  on  computed  structural  and  functional  representation  of  time-lapse 
video  images. 


4.2  Segmentation  of  single-channel  data 

Our  current  production  use  of  segmentation  algorithms  is  limited  to  nuclei  observed  with  fluores¬ 
cence  dye  using  two  different  techniques.  The  first  approach  is  based  on  a  variation  of  Hough 
transform  that  is  suitable  for  a  large  class  of  cells  as  long  as  they  conform  to  a  specifically  pa¬ 
rameterized  geometric  model.  The  second  approach  focuses  on  extraction  of  partial  information 
corresponding  to  step  and  roof  edges  and  then  groups  them  through  geometric  reasoning  [13,  14]. 
The  intent  is  to  construct  a  partitioning  that  is  globally  consistent.  A  list  of  step  edges  is  con¬ 
structed  by  thresholding  raw  images,  recovering  corresponding  boundaries,  and  localizing  concave 
corners  from  their  polygonal  approximation.  These  corners  provide  possible  cues  to  the  position 
where  two  adjacent  nuclei  may  touch  each  other.  Thresholding  separates  big  clumps  consisting  of 
several  nuclei  squeezed  together,  but  the  boundaries  between  adjacent  nuclei  cannot  be  detected 
because  they  have  higher  intensities  than  background.  The  partial  boundaries  between  adjacent 
nuclei  can  be  recovered;  however,  by  extracting  crease  segments  [24,  40,  39],  as  shown  in  Figure 
3(d).  These  crease  segments  correspond  to  trough  edges  (positive  curvature  maxima),  but  false 
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creases  may  also  be  extracted  in  the  process.  Nevertheless,  dealing  with  noise  and  erroneous  seg¬ 
ments  is  a  higher-level  process  that  is  handled  during  the  grouping  process.  Our  system  generates 
a  number  of  hypotheses  for  possible  grouping  of  the  boundary  segments.  A  unique  feature  of  this 
system  is  in  hyperquadric  representation  of  each  hypothesis  and  use  of  this  representation  for  global 
consistency.  The  main  advantage  of  such  a  parameterized  representation  (as  opposed  to  polygonal 
representation)  is  compactness  and  better  stability  in  shape  description  from  partial  information. 
In  this  fashion,  each  step-edge  boundary  segment  belongs  to  oire  and  only  one  nucleus  while  each 
trough-edge  boundary  segment  is  shared  by  two  and  only  two  nuclei.  These  initial  hypotheses  and 
their  localized  inter-relationship  provide  the  basis  for  search  in  the  grouping  step.  This  is  expressed 
in  terms  of  an  adequate  cost  function  and  minimized  through  dynamic  programming.  The  final 
result  of  this  computational  step  is  then  shown  to  an  operator  for  verification  and  elimination  of 
false  alarms.  An  example  is  shown  in  Figure  4,  where  comparison  to  manual  segmentation  is  shown. 
A  major  limitation  of  this  approach  is  its  computational  complexity.  Grouping  of  local  features  is 
a  complex  process,  and  simpler  techniques  are  highly  desirable. 


(a)  (b)  (c)  (d) 


Figure  3:  This  example  demonstrates  that  segmentation  of  nuclei  with  purely  global  or  local  oper¬ 
ations  is  not  feasible;  (a)  original  image;  (b)  threshold  image;  (c)  boundary  of  nuclei;  and  (d)  local 
troughs. 


(a)  (b)  (c) 

Figure  4:  Segmentation  results:  (a)  original  image;  (b)  manual  segmentation;  (c)  our  results  on 
segmentation  of  nuclei. 

Recently,  we  have  developed  three  new  techniques.  One  is  based  on  a  new  regularized  form, 
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the  second  on  a  vector  representation  of  the  underlying  image  followed  by  feature  extraction  in 
vector  space,  and  the  third  on  a  geometric  and  texture  model  of  a  cell  and  its  interaction  with 
neighborhoring  cells.  The  new  regularization  form  has  its  root  in  visualization  of  3D  objects: 
however,  we  have  also  found  it  to  be  applicable  to  segmentation  of  scientific  images.  The  vector 
representation  of  images  has  its  root  in  analysis  of  spatio-temporal  satellite  images  [41,  42]. 

I 

4.2.1  A  regularized  edge  detector  based  on  supremum 

Current  regularization  techniques  attempt  to  minimize  an  integral  such  as  V /P-  this 

formulation  has  no  control  on  the  local  property  of  /.  In  other  words,  the  global  average  of  |  V  /I 
may  be  small,  but  locally  /  may  change  sharply.  A  way  to  overcome  this  issue  is  to  minimize  |  y/i 
at  every  point,  which  leads  to  minimization  of  the  supremum  of  |  y  /|.  It  is  natural  to  consider  the 
functional  If(/)  =  y  /|  as  a  “limit”  of  the  sequence  of  functionals 

ffN(f)  =  (/^iV  A  =  1, 2,3, ...  (1) 

The  Euler  equation  for  the  minimization  of  the  functional  can  be  expressed  as: 

+fxfxx  +  ^fxfyfxy  +  fyfyy}  =  0  (2) 

where  a  subscript  indicates  a  derivative,  such  &s  =  %,fxy  =  By  removing  the  first 

coefficient  and  letting  N  oo,  \ye  have 

fxfxx  +  ^fxfyfxy  +  fyfyy  ~  0  (3) 

Equation  (3)  is  called  the  Infinite  Laplacian  Equation  (ILE),  which  has  been  studied  widely  in  the 
literature  [1,  2,  3,  15].  Some  important  properties  of  the  equation  [2]  are:  (1)  there  is  at  most  one 
solution  and  if  we  redefine  the  “solution”  in  a  suitable  weak  sense,  then  a  continuous  solution  to 
does  exist;  (2)  the  trajectory  of  the  gradient  of  /  is  either  a  convex  curve  or  a  straight  line;  and 
(3)  there  are  no  stationary  points  |  y  /]  =  0  in  i?. 

The  rationale  for  optimality  of  the  supremum  is  based  on  the  “equal  importance  criterion” 
[11,  12].  This  criterion  asserts  that  every  point  in  R  is  equally  important  and  contributes  similarly 
to  the  estimation  process.  Any  other  assumption  means  that  we  need  to  know  some  additional 
information  about  the  curve.  It  is  easy  to  verify  that  ILE  is  equivalent  to  the  following  equation: 

J'(/)  =  V(IV/r)-|^  =  0  (4) 

which  implies  that  along  each  trajectory  of  the  gradient  of  /,  the  magnitude  of  the  gradient  is  a 
constant.  Figure  5  indicates  that  minimizing  the  integral  of  |  y  /|  aims  to  minimize  the  area  while 
minimizing  the  supremum  aims  to  minimize  the  maximum.  Although  the  overall  integral  of  |  y  /] 
may  be  larger,  the  supremum  is  smaller  and  the  gradient  is  more  likely  to  concentrate  at  a  smaller 
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range.  It  is  interesting  to  note  that  Equation  3  is  equivalent  to  the  zero-crossing  {fxx  +  fyy  =  0)  in 
the  local  coordinate  system,  where  the  local  coordinate  system  is  defined  by  v/- 


).t 


Igradf) 


mia.  Energy  =  min.  sup.jgrad  ^ 


X 


(a)  (b) 

Figure  5:  Comparison  of  our  method  with  traditional  regularization  terms;  (a)  solution  to  mini¬ 
mization  of  gradient;  (b)  solution  to  minimization  of  supremum. 

The  edge  detection  results  reveal  closed  regions.  These  regions  correspond  to  spatial  blobs  that 
are  further  filtered  for  size,  response,  and  convexity  constraint.  An  example  is  show  in  Figure  6. 
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Figure  6:  Segmentation  result  of  human  epithelial  cells  at  low  resolution:  (a)  original  image;  (b) 
extracted  nuclei. 


12 


Annual  Report:  Callahan,  Parvin  Page  134 


4.2.2  Tensor-based  feature  analysis 


Vector  field  analysis  is  a  well-studied  technique  in  computer  vision  and  pattern  recognition  as  well  as 
other  classic  fields  such  as  mathematics  and  physics.  Many  concepts  and  tools  have  been  developed  I 
for  motion  based  applications.  Our  work  extends  the  current  state  of  the  art  to  segrhentation  and 
interpretation  of  scientific  images.  Given  an  intensity  image  there  exists  a  natural  vector 

field  derived  from  the  intensity  image,  which  corresponds  to  the  gradient  field  rotated  by  7r/2: 


V 


=  (- 


dfo  dfo 

dy'  dx’ 


The  problem  here  is  that  the  vector  field  v  is  noisy,  and  some  type  of  regularization  needs  to  be 
introduced.  This  can  be  expressed  as 


min  J  J \\u-v\\‘^  +  a‘^\\‘Vu\\‘^dxdy 

where  u  is  the  regularized  vector  field.  An  elliptic  PDE  can  be  solved  iteratively.  A  first  order 
approximation  can  be  computed  by  a  simple  linear  (Gaussian)  scale  space  model  [23]: 


fo{-) 


where 

g{x,y,t)  =  —e  *> 

is  the  Gaussian  kernel  with  standard  deviation  a  =  \/^. 

Singularities  of  the  vector  field  then  provide  a  compact  abstraction  for  the  dense  vector  rep¬ 
resentation.  These  singularities  cd,n  be  characterized  by  point  or  linear  features.  Point  features 
include  vortices  and  saddle  points.  An  example  of  such  a  vector  field  and  its  singularities  (vortices) 
on  a  synthetic  scene  is  shown  in  Figure  7.  Once  these  patterns  are  detected,  the  corresponding 
objects  can  be  extracted.  Rao  [34]  used  local  Jacobian  for  feature  extraction  from  the  underlying 
vector  field  to  detect  singularities.  Our  approach  is  more  robust  and  provides  a  measure  for  the  size 
of  the  vortex  as  weU.  The  vortex  size  complements  localization  of  convex  blobs  eventhough  they 
are  connected  (or  touching  one  another).  This  is  based  on  Jordan  index  to  localize  singularities 
in  the  underlying  vector  field.  Let  F  =  {u,  n)  be  a  vector  field  and  J  be  a  Jordan  curve  with  no 
critical  point  on  it.  The  index  of  J  is  defined  by 


lndex(./)  =  —  (f 
'2-k  Jj 


udv  —  vdii 
iF  -f 


At  each  point  P,  we  choose  a  small  circle  of  radius  R  (denoted  by  Jp)  around  P  and  compute 
Index(Jp).  The  flow  field  (u,  n)  can  then  be  classified  according  to: 

1.  The  index  of  a  vortex  is  equal  to  -f  1  (the  classification  of  singular  points  in  a  vector  field  is 
given  in  [34]),  and 

2.  The  index  of  a  saddle  point  is  equal  to  —1. 
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Note  that  there  is  no  node  in  the  vector  field  because  its  divergence  is  zero  everywhere,  i.e., 

j.  d  ,  Of,  d  df 


I, 

The  vortex  size  is  then  estinaated  by  a  simple  search  technique.  If  a  point  a  (x,  y)  is  a  vortex,  then 
its  size  jR*(x,  y)  can  be  defined  as 

R*(x,y)  =  ma.x{R\lndex(jl^y^)  =  1} 


In  other  words,  R*{x,y)  is  the  largest  R  such  that  the  index  of  remains  to  be  1.  Being  an 
integral  of  the  first  order  partial  derivatives,  the  Jordan  index  is,  in  some  sense,  a  function  of  the 
intensity  image.  In  contrast,  other  techniques  are  based  on  high  order  derivatives,  which  are  bound 
to  be  more  noisy.  Figures  7,  8,  and  9  show  the  result  of  vortices  and  region  segmentation  on  a 
synthetic  image,  nuclei  labeled  with  fluorescent  dye,  and  cells  observed  with  transmission  light. 
The  later  result  is  significant  because  current  techniques  in  localizing  individual  cells  are  limited  to 
fluorescent  imaging,  which  adds  a  layer  of  complexity  to  the  design  of  dynamic  experiments. 


(a)  (b)  (c)  (d) 

Figure  7:  Vector  field  analysis  on  a  synthetic  image:  (a)  original  image;  (b)  vector  field  correspond¬ 
ing  to  the  gradient  field;  (c)  detection  of  vortices  from  the  vector  field;  (d)  localization  results. 


4.2.3  Geometric  and  texture  model 

Although  the  previous  techniques  are  applicable  for  localizing  nuclei,  a  more  advanced  technique  is 
needed  for  a  more  detailed  segmentation  at  higher  imaging  resolution.  This  is  based  on  a  geometric 
model  of  the  cell,  its  texture  in  various  compartments,  and  relationship  of  its  features  with  respect 
one  another.  The  geometric  model  is  expressed  as  a  graph,  and  the  texture  is  computed  using  an 
array  of  orientation  and  scale  specific  filters.  These  local  feature  activities  help  collect  an  ensemble 
of  information  that  is  further  refined  with  dynamic  programming  [28].  Our  preliminary  result, 
shown  in  Figure  10,  indicates  reasonable  segmentation  of  nucleus  and  cytoplasm,  which  can  be 
used  as  a  basis  for  computing  responses  in  each  of  these  compartments.  Furthermore,  responses 
can  be  annotated  with  a  proximity  filter  (distance  transform)  to  reveal  whether  responses  in  the 
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Figure  8:  Detection  of  vortices  from  nuclei  of  MCF7  cells:  (a)  original  image;  (b)  fields  of  vortices. 


(a)  (b) 


Figure  9:  Detection  of  cells  from  phase  contrast  imaging:  (a)  the  original  image  with  detected 
vortices;  (b)  cell  boundaries  from  a  balloon  expansion. 

1.5 
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cytoplasm  have  a  preferred  localization  or  are  diffused.  For  example,  it  is  important  to  know 
whether  the  concentration  of  a  particular  compound  is  at  lysosome  or  mitochondrial.  In  general, 
preferred  uptake  at  lysosome  tends  to  be  of  less  interest. 


Figure  10:  Preliminary  result  for  segmentation  of  MCF7  cells  at  mid-resolution  uses  geometric  and 
texture  features  to  delineate  sub-cellular  structures.  The  functional  response  in  each  compartment 
can  then  be  measured  for  large  scale  statistical  analysis  as  well  as  visual  servoing  in  real-time. 


4.3  Segmentation  of  multi-channel  data 

Imaging  mode  may  be  multispectral.  In  this  context,  a  sample  is  imaged  with  different  dyes  and 
filters  to  accentuate  different  cellular  components.  Here,  the  system  must  be  able  to  analyze  sev¬ 
eral  channels  of  data  simultaneously,  select  a  specific  location  in  the  cytoplasmic  region,  and  make 
repeated  measurements.  An  efficient  approach  to  fusing  images  based  on  a  Bayesian  framework 
has  been  developed.  This  approach  can  also  be  extended  to  3D  (with  confocal  microscopes),  as 
demonstrated  by  our  earlier  work  [29].  The  purpose  of  data  fusion  is  complementary  processing  of 
different  modalities  for  segmentation  and  labeling.  From  this  perspective,  the  segmentation  proce¬ 
dure  should  label  each  pixel  in  the  data  volume  accordingly.  A  number  of  ambiguities;  however, 
can  complicate  the  labeling  process.  These  ambiguities  can  arise  from  purely  local  processing  and 
the  absence  of  any  high  level  feedback.  The  sources  for  the  ambiguities  include  corruption  of  data 
by  noise,  performance  limitation  of  algorithms  for  extracting  local  features,  and  the  existence  of 
nonessential  features  that  impede  the  labeling  task.  One  aspect  of  region  segmentation  is  estimat¬ 
ing  the  average  intensity  of  each  region  by  analyzing  the  histogram.  The  initial  position  of  the 
peaks  in  the  histogram  are  approximated  and  then  refined  with  least  square  approximation.  The 
next  step  of  the  computational  process  is  to  use  these  peaks  as  cluster  centers  to  enforce  local 
consistency  in  the  image  space.  Here,  a  Bayesian  framework  is  used  to  label  images  based  on  their 
multiple  channel  information.  Specifically,  we  use  a  Bayesian  hierarchical  model  with  three  levels 
of  hierarchy.  The  first  level  is  a  model  for  the  underlying  classification,  Z.  Under  ideal  conditions, 
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observing  Z  under  M  different  modalities  (here,  there  are  three  fluorescent  images  correspond  to 
360nm,  490nm,  and  570nm  fluorescent  emission,  respectively)  specifies  an  ideal  representation,  X,-, 
of  the  data  that  are  corrupted  by  different  types  of  noise  in  the  imaging  system.  The  third  level  of 
the  hierarchy  corresponds  to  the  actual  observation  of  data  1;.  The  first  level  uses  Markov  Random 
Field  (MR.F)  with  prior  probability  density  function  in  the  form  of  Ising  model  with  a  positive  pa¬ 
rameter  (3.  This  parameter  encourages  cooperation  among  nearby  pixels.  In  a  Bayesian  framework, 
yS  may  assume  prior  ignorance  about  the  scene  content.  The  second  level  models  X  as  conditionally 
independent  given  Z,  i.e.,  P(Xi,  ...,Xm\Z)  =  P{Xi\Z)...P{Xm\Z).  This  is  reasonable  because  if 
a  classification  is  labeled  as  cytoplasm,  then  the  ideal  response  of  the  green  channel  (cytoplasm) 
and  the  blue  channel  (nuclei)  will  not  affect  each  other.  Finally,  the  third  level  of  hierarchy,  Yj-,  is 
modeled  as  a  Gaussian  distribution.  As  a  result,  Yi  will  be  a  locally  blurred  representation  of  X,-. 
The  full  model  can  be  written  as 

P{Z,Xi\Yi)  =  Y[P{Y,\Z)Y[P{X,\Z)P{Z)  (5) 

We  use  maximum  a  posteriori  (MAP)  estimates  to  obtain  X,-  and  Z.  Because  MAP  computation  is 
inherently  infeasible,  we  use  numerical  approximation  based  on  iterative  conditioning  modes  (ICM) 
[5]  to  find  the  local  optimum.  An  example  of  three  channel  fluorescent  image  analysis  is  shown  in 
Figure  11;  .  • 


•  (a)  (b)  (c) 

Figure  11:  Multiple  channel  data  fusion  for  segmentation  of  nuclei:  (a)  DAPI  image  (blue  channel); 
(b)  Texas  Red  image  (red  channel);  (c)  segmentation  results  on  FITC  (green  channel). 


5  Research  design  and  methods 

This  section  outlines  new  developments  to  accommodate  specific  experiments. 

5.1  Detailed  segmentation 

Although  the  group  has  a  wide  array  of  computational  tools  for  automated  analysis  of  images,  these 
tools  are  not  sufficient  for  detailed  analysis  of  various  compartments  within  the  cell.  Specifically, 
some  form  of  image  restoration  and  a  better  segmentation  model  are  Needed.  The  requirement  for 
image  restoration  originates  from  the  fact  that  there  could  be  potential  cross-talk  between  responses 
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obtained  at  different  focal  planes.  Furthermore,  as  Figure  10  shows,  different  compartments  in  a 
cell  may  not  have  the  necessary  clarity  and  focus  at  a  given  focal  length.  We  will  incorporate  a 
simple  nearest  neighbor  deconvolution  to  reduce  cross-talk,  noise,  and  improve  the  required  image 
quality  and  resolution.  Segmentation  must  be  more  detailed  (e.g.,  with  respect  to  localizing  nucleus, 
membrane  bounding  the  cytoplasm,  mitochondrial,  and  lysosome).  A  precise  structural  definition 
of  these  compartments  is  necessary  to  measure  how  (fluorescently)  labeled  compounds  are  expressed 
in  each  compartment  and  whether  such  expressions  are  diffused  in  cytoplasm  or  highly  localized. 
This  will  be  achieved  by  designing  a  strategy  to  couple  on-line  segmentation  (analysis  of  images) 
with  control  of  the  excitatory  filters  and  perfusion  of  specific  dye  into  a  petri  dish.  The  dye  labels 
a  specific  area,  in  the  cell  that  is  visible  at  a  particular  excitation  filter.  This  is  another  form  of 
visual  servoing  in  which  the  instrument  is  placed  in  a  particular  state,  based  on  computed  features 
from  image  content.  We  will  build  a  real-time  computational  toolkit  to  track  the  trajectory  and 
expression  of  nakid  antisense  compounds  in  subcellular  compartments.  The  computed  tracking 
results  will  then  be  used  to  populate  the  database. 

5.2  Feature  based  image  storage 

Our  current  model  of  storage  are  based  on  flat  files  that  is  difficult  to  search  and  maintain.  We 
propose  to  leverage  an  active  bioinformatic  program  to  build  the  required  schema  for  storage 
handling.  The  design  of  current  schema  for  low  dose  radiation  study  is  shown  in  Figure  12.  This 
design  will  be  extended  to  accommodate  VSOM  and  the  required  chemistry.  The  intent  is  to  build 
a  feature  based  spatio-temporal  database  that  will  be  used  for  constructing  a  model  of  how  and 
where  each  AS-ODN  compound  settles  in  subcellular  compartment  and  what  it  takes  to  free  it.  The 
proposed  multidimensional  database,  shown  in  Figure  1.3,  will  represent  the  structure  and  function 
of  subcellular  anatomy,  the  dose  response  curve  for  each  compartment,  and  the  trajectory  of  the 
compounds  from  one  compartmenfto  next.  Each  experiment  consists  of  a  target  frame  (image)  that 
corresponds  to  a  specific  physical  location  on  the  petri  dish.  The  subcellular  structures  in  the  cell 
are  segmented  and  stored,  as  an  attribute  graph  along  with  the  annotation  data.  The  target  frame 
is  then  observed  with  time-lapsed  video  microscopy  while  compounds  are  being  injected  into  petri 
dish  under  computer  control.  The  response,  in  each  compartment,  is  recorded  and  the  trajectory 
of  the  antisense  compound  is  tracked  as  it  migrates  in  the  subcellular  compartment. 

The  functional  view  of  the  stored  data  wiU  be  hierarchical.  At  the  lowest  level,  raw  images  and 
the  corresponding  physical  annotation  of  images  will  be  stored.  At  the  next  level,  the  attributed 
graph  model  (structure),  corresponding  responses  (function),  and  dynamic  annotation  of  environ¬ 
mental  condition  (e.g.,  compound  type,  flow  rate)  will  be  stored.  At  the  next  level,  the  temporal 
expression  of  each  compartment  in  a  cell  will  be  summarized  as  a  response  curve  along  with  tra¬ 
jectory  of  the  compound.  These  responses  may  not  be  similar  for  all  cells  under  observation.  We 
expect;  however,  that  these  response  curves  will  form  clusters,  with  each  cluster  corresponding  to  a 
group  of  cells  in  similar  state  or  cell  lines  of  a  particular  origin.  These  clusters  must  be  identified  for 
subsequent  correlation  studies.  Each  cluster  corresponds  to  a  time  series  event  that  correspond  to  a 
particular  compartment  in  the  cells.  The  average  behavior  of  these  responses  in  each  cluster  will  be 
represented  through  single  value  decomposition.  Formally,  the  observed  response  is  a  multivariable 
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'Figure  12:  Current  schema  model  for  low  dose  radiation  studies  (funded  through  DOE-OBER) 
function  defined  as 

ft{Channel,  CellType,  Compartment,  Dye)  -  f{[Dye],  -,T)  (6) 

Where  /  is  the  mean  intensity  observed  in  a  specific  compartment  at  time  t,  [Dye]  is  the^com 
centration  of  dye,  Channel  is  the  position  of  the  excitation  filter,  T  is  temperature,  and 
corresponds  to  changes  in  the  flow  of  compounds  being  injected.  Schema  will  be  developed  to 
capture  this  time  varying  information  to  store  the  observations  into  a  database  for  subsequent 
knowledge  discovery.  Each  experimental  cycle  generates  a  fingerprint  for  each  observed  cell.  These 
individual  responses  provide  a  compact  representation  of  intra-cellular  activities  in  the  image  space 
that  can  be  visualized  per  cell,  population  of  cells,  cell  tj'pes,  etc. 


5.3  Learning  from  cell  responses 

We  will  extend  and  apply  learning  techniques  to  help  refine  and  design  new  antisense  compounds 
for  targeting  mRNA  sequence  into  cytoplasmic  or  nucleoplasmic  compartments  of  the  cell.  In  this 
context,  computed  responses  together  with  their  organization  and  trajectories  aid  in  development 
of  a  model  and  environment  in  which  subsequent  stepwise  search  for  design  and  modification  of 
antisense  can  take  place.  The  basis  for  any  learning  approach  is  a  policy,  rewards,  values,  and  a 
model  of  environment.  One  purpose  of  model  reconstruction  is  to  evaluate  various  policies  and 
reward  functions.  For  example,  what  are  the  transition  probabilities  that  describe  the  state  of  a 
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Figure  13:  Feature  based  hierarchical  storage  of  computed  structure  and  function  in  each  com¬ 
partment  of  the  cell.  Each  experiment  consists  of  time  lapsed  video  imaging  and  the  required 
manipulation  of  perfusion  pumps  (not  shown  here).  Images  are  stored  at  the  lowest  level.  The  next 
level  stores  the  structure  and  function  of  the  subcellular  anatomy  as  an  attributed  graph.  The  third 
level  constructs  the  corresponding  response  curves  at  a  specific  location  and  inferred  trajectories 
of  the  compound.  The  fourth  layer  provides  a  mechanism  to  construct  the  transition  probabilities 
together  with  cross-correlation  studies  between  experiments. 
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cell  as  AS-ODN  is  injected  or  pH  altered  in  its  environment?  Or  what  should  be  the  modulation 
frequency  of  a  compound  (alternating  high/low  concentration)  to  track  the  retention  of  a  particular 
compound  in  the  cell  structure?  This  step  provides  the  basis  for  construction  of  a  model  and  envi¬ 
ronment.  Furthermore,  we  expect  that  such  a  model  will  vary  from  one  cell-line  to  ^nother.  These 
variations  are  significant  by  their  own  merit  and  when  quantified,  can  serve  as  a  valuable  valida¬ 
tion.  The  content  of  a  database  will  provide  the  basis  for  evaluation,  refinement,  and  localization 
of  similarity  measures.  Similarities  will  be  measured  by  analyzing  clusters  on  the  basis  of  state 
and  action.  These  clusters  aid  in  computation  of  co-occurrence  statistics  over  different  responses 
for  establish  an  equivalence  classes.  Alternatively,  a  cell  response  and  its  corresponding  attributes 
(dynamic  annotations)  will  generate  a  representation  within  a  sequence.  This  representation  can 
then  be  used  to  construct  joint  co-occurrence  statistics  in  that  sequence  together  with  transition 
probabilities.  The  co-occurrence  statistics  will  aid  in  hypothesizing  a  binary  tree  classification  of 
an  eixsemble  of  representative  features.  One  utility  of  this  method  is  in  detecting  and  classify¬ 
ing  correlated  sequences.  Furthermore,  once  a  model  is  established,  co-occurrence  statistics  will 
provide  a  means  of  detecting  outliers.  The  transition  probabilities  will  provide  the  knowledge  of 
environment,  which  can  be  used  for  reinforcement  learning.  In  addition,  we  will  explore  modeling 
the  cell  responses  as  a  time  varying  linear  system.  Linear  systems  are  simple  and  powerful  tech¬ 
niques  to  characterize  the  behavior  of  a  system.  In  this  coixtext,  a  state  space  representation  of  the 
system  with  variable  memory  (delay)  is  constructed  and  parameters  of  the  associated  matrices  are 
estimated  and  validated. 


5.3.1  Reinforceiixeixt  learning 

The  intent  is  to  use  prior  knowledge  as  a  starting  point  to  learn  how  to  manipulate  an  antisense 
for  guiding  it  into  a  specific  site'  in  a  systematic  way.  This  is  often  referred  to  as  “reinforcement 
learning”  in  the  literature,  which  may  be  modeled  as  a  Markov  decision  process  (MDP).  A  partic¬ 
ular  MDP  is  defined  by  its  state  (observed  response)  and  action  set  that  includes  modification  of 
antisense  by  changing  its  binding  constant,  altering  the  concentration  of  important  intra-cellular 
ions  for  releasing  sequestered  antisense  from  nucleus,  etc.  The  reward  in  such  a  system  is  measured 
as  the  chaixges  in  computed  response  or  its  episodic  gradient.  Furthermore,  the  transition  proba¬ 
bilities  (of  the  Markov  decision  process)  can  be  computed  from  observations  made  from  the  same 
assay  in  the  database.  These  transition  probabilities  provide  the  enabling  technology  for  an  opti- 
ixxal  policy  (schedule).  In  general,  such  a  policy  can  be  expressed  as  dynamic  programming,  Monte 
Carlo  methods,  and  temporal  difference  learning.  Dynamic  programixxing  requires  a  model  of  cell 
response.  Such  a  model  can  be  hypothesized  from  the  database.  Presumably,  as  the  database  ex¬ 
pands,  models  may  become  more  accessible.  Monte  Carlo  methods  do  not  require  a  model,  but  they 
are  not  suitable  for  step-by-step  incremental  computation.  On  the  other  hand,  temporal  methods 
require  no  model  and  are  fully  incremeixtal.  The  PI  has  extensive  experience  in  expressing  a  policy 
based  on  dynamic  programming  [14,  28,  32].  Still,  all  three  methods  apply  to  our  research  agenda. 
For  example,  Monte  Carlo  techniques  can  be  used  in  one  incremental  step  (or  one  episode)  if  sample 
transition  probabilities  (not  a  complete  one)  exist.  And  temporal  differencing  is  the  combination 
of  Monte  Carlo  and  dynamic  programming  (i.e.,  learning  directly  from  raw  experience  without  the 
model,  and  then  updatiixg  estimates  ba.sed  on  currently  learned  estimates). 
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5.4  Experiments 


Our  experimental  plan  is  designed  to  complement  the  companion  grant,  “Strategies  for  Imaging 
Gene  Expression  with  PET”,  submitted  by  S.  Taylor,  et  al.  from  the  LBNL  Center ^for  Functional 
Imaging  (LBNL  CGI)  in  which  in  vivo  animal  experiments  and  PET  imaging  will’be  performed 
using  radiolabeled  AS-ODN.  We  plan  to  collaborate  closely  with  this  group,  and  for  this  reason  we 
will  use  the  same  cell  line,  (BT-  474  cells,  breast  cancer  cells  that  overexpress  the  apoptotic  bcl-2 
protein)  and  the  same  three  AS-ODN  sequences  designed  to  target  the  bcl-2  protein.  Our  in  vitro 
technology  will  allow  us  to  test  more  modifications  of  these  compounds  than  is  possible  in  in  vivo 
studies  of  nude  mice  bearing  BT-474  tumor  xenografts.  However,  many  of  our  joint  experiments  will  . 
include  AS-ODN  compounds  of  the  same  sequence  with  the  same  modifications,  with  the  exception 
of  the  modification  at  the  5’  terminus  of  the  AS-ODNs,  which  be  a  fiuorochrome  in  the  case  of  our 
experiments  but  will  be  one  of  two  radiofluorinated  moeities  for  in  vivo  PET  imaging  studies.  In 
both  CcLses,  however,  a  hexyl-amine  linker  will  be  attached  to  the  5’-terminus  to  facilitate  attachment 
of  the  fiuorochrome  and  radiolabel.  Our  proposal  and  the  Taylor  proposal  are  also  integrated  in 
several  other  ways.  For  example,  the  two  groups  will  conduct  control  experiments  using  the  same 
“sense”  sequences  that  are  complimentary  to  the  three  AS-ODN  sequences  we  have  chosen,  and 
the  same  non-bcl-2  producing  cell  line.  Further,  in  the  Taylor  grant,  differential  cell  sedimentation 
centrifugation  experiments  and  binding  assays  using  the  BT-474  cell  line  will  be  performed  with 
both  radiolabeled  and  fiuorescence  labeled  compounds.  In  addition,  both  groups  will  perform 
studies  to  determine  if  the  delivery  of  AS-ODN  is  enhanced  using  liposomes  and  immunoliposomes. 
The  same  liposome  preparations  will  be  used  between  our  two  groups.  For  our  biological  endpoint 
studies,  we  will  in  some  cases  use  a  non-fiuorescent,  non-  radioactive  version  of  the  fiuorinated 
moiety  used  in  the  radiolabel  studies.  In  this  manner,  we  will  determine  the  consequences,  if  any, 
of  different  substituents  at  the  5’-terminus  of  the  AS-ODNs.  It  will  also  be  possible  for  our  group 
to  determine  the  amount  of  signal  amphfication  produced  by  the  /?-galactosidase  amplification 
protocol  proposed  by  the  Taylor  group.  AS-ODN  sequences  to  be  used: 

1.  PT-G3139:  TCTC  CCAG  CGTG  CGCC  AT 

The  full  phophorothiate  of  this  18-mer  with  sequence  antisense  to  the  first  six  codons  of  the 
open  reading  frame  of  bcl-2  has  shown  efficacy  against  the  DoHH2  lymphoma  implanted  in 
severe  immunodeficient  mice  (Raynaud  et  al.,  1997). 

2.  PNA-1:  CCCC  AGCC  CCTA  CCC 

3.  PNA-4:  AGCC  TGCG  CCAT  CCC 

Peptide  nucleic  acid  (PNA)  is  a  modified  oligonucleotide  in  which  the  entire  deoxy-ribose 
backbone  has  been  replaced  with  a  polyamide  (peptide)  chain  (Gewirtz  et  al.,  1998  [17]; 
Temsamani  and  Guinot,  1997  [36]).  These  two  15-mer  PNA-ODNs  have  been  evaluated  in 
vitro  in  a  cell-  free  system  (Mologni  et  al.,  1999).  Of  particular  interest  is  the  fact  that  a 
complete  block  of  niRNA  translation  is  only  achieved  when  both  PNA-  ODNs  are  present 
simultaneously. 
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5.4.1  VSOM  experiments 

Flourescence  labeled  versions  of  PT-G3139,  PNA-1,  and  PNA-4  will  be  obtained  commercially 
(Genset,  http://www.gensetoligos.com)  and  from  the  Taylor  group.  We  will  also  obtain  modifica¬ 
tions  of  these  AS-ODNs  where  the  backbone  consists  entirely  of  PD,  PT,  MP,  or  F^iNA  backbone 
linkages.  In  addition,  we  will  obtain  compounds  with  various  permutations  and  mixtures  of  back¬ 
bone  linkages. 

Initial  experiments  will  be  performed  and  recorded  in  the  database  as  a  series  of  single-cell 
responses.  We  will  use  a  set  of  fluorescently  labeled  AS-ODNs  with  various  modifications  of  the 
backbone  linkages,  as  described  above.  A  set  of  baseline  VSOM  runs  will  be  performed.  In  these 
runs,  cells  will  be  grown  on  microscope  coverslips,  and  a  63X,  NA  1.3  oil-immersion  objective  will 
be  used.  Cells  will  be  placed  in  a  temperature  controlled  microperfusion  chamber  containing  cells  at 
37  ?C.  An  initial  segmentation  of  the  cytoplasmic,  nucleoplasmic,  and  mitochondrial  compartment 
will  be  performed  in  the  following  manner.  A  computer  controlled  syringe  will  inject  the  fluorescent 
compound  TMRE  contained  in  a  D-PBS  (Dulbecco’s  phosphate-buffered  saline)  buffer  containing 
calcium,  magnesium,  glucose  and  pyruvate.  The  compound  TMRE  (tetramethyl  rhodamine  ethyl 
ester.  Molecular  Probes,  Eugene,  OR)  is  a  cationic  redistribution  dye  used  to  determine  the  mito¬ 
chondrial  membrane  potential  according  to  the  Nernst  equation  (Farkas  et  ah,  1989).  Unlike  the 
mitochondrial  dye  rhodamine  123,  TMRE  equilibrates  rapidly  and  reversibly  into  the  mitochon¬ 
dria  and  (to  a  lesser  extent)  the  cytoplasm.  It  does  not  stain  the  nucleoplasmic  compartment. 
Our  ability  to  perform  such  segmentation  on  TMRE  stained  MCF-7  cells  (a  human  breast  cancer 
cell  line)  was  shown  in  Figure  10.  Next  TMRE  will  be  flushed  out  of  the  cell  using  a  computer- 
controlled  syringe  containing  buffer  only.  TMRE  is  readily  rinsed  out  of  the  cell.  We  will  then 
proceed  to  inject  fluorescently  labeled  AS-ODN  into  the  microperfusion  chamber  using  yet  another 
computer  controlled-  syringe.  The  concentration  at  which  fluorescent  signal  from  the  AS-  ODN 
becomes  visible  will  be  noted,  and' the  AS-ODN  perfusion  will  be  automatically  halted.  This  will 
complete  the  first  half  of  a  VSOM  run. 

It  should  be  noted  that  all  digital  images  will  become  part  of  the  database  so  that  they  make  be 
replayed  in  order  to  verify  VSOM  operation.  Thus,  multi-channel  digital  images,  experimental  pa¬ 
rameters,  and  single-cell  responses  (5-10  cells  at  63X)  to  computer-controlled  compound  perfusions 
will  become  part  of  the  database. 

The  second  half  of  these  initial  VSOM  runs  will  be  conducted  as  follows.  The  fluorescence 
distribution  pattern  of  the  fluorescently  labeled  AS-  ODN  will  be  compared  with  either  (i)  previously 
observed  fluorescence  distribution  patterns  obtained  for  the  same  ceU  type  using  organelle-  specific 
fluorescence  dyes  (Molecular  Probes,  http://www.probes.com/)  or  (ii)  the  existing  fluorescence 
pattern  in  a  separate  fluorescence  channel  for  the  current  cells,  dual-stained  with  both  fluorescent 
AS-ODN  and  an  organelle-specific  fluorescence  dye.  After  the  VSOM  makes  a  determination  of 
the  AS-ODN  location  within  the  cell,  it  will  apply  the  appropriate  compound  from  the  appropriate 
syringe  and  observe  the  effect  on  AS-ODN  distribution  within  the  cell. 

In  the  case  of  lysosomal  sequestration  of  fluorescently  labeled  AS-ODN,  compounds  known  to 
selectively  permeablize  lysosomes  will  be  perfused  into  the  chamber  at  a  controlled  rate  by  the 
computer-controlled  perfusion  pumps.  Several  compounds,  such  as  the  sodium  proton  ionophore 

23 


Annual  Report:  Callahan,  Parvin  Page  145 


monesin,  have  been  demonstrated  to  free  fluorescent  compounds  trapped  in  the  acidic  vesicles  of  of 
human  breast  cancer  cells  (Schindler  et  ah,  1996).  The  amount  of  monesin  (for  example),  require 
to  free  fluorescently  labeled  AS-ODNs  will  be  logged  in  the  database.  The  time  for  redistribution 
to  the  next  compartment  (the  nucleus  for  example)  and  the  resulting  pattern  of  jtaining  in  the 
nucleus  will  also  be  recorded. 

VSOM  perturbations  to  dislodge  AS-ODN  compounds  from  the  nucleus  will  be  performed  using 
’’clamping”  techniques  which  allow  the  experimentalist  to  vary  the  concentration  of  intracellular 
ions  such  as  potassium  by  simply  varying  the  potassium  concentration  of  the  surrounding  extra¬ 
cellular  medium  (Negulescu  et  ah,  1994).  These  same  methods  can  be  used  to  alter  levels  of 
intracellular  calcium.  The  VSOM  system  will  ramp  up  the  concentrations  of  such  ions  within  the 
cell  in  an  attempt  to  interfere  with  the  binding  of  AS-ODN  to  basic  nuclear  proteins.  Once  again 
the  concentrations  and  other  environmental  conditions  necessary  to  dislodge  the  compounds  from 
the  nuclear  compartment  will  be  recorded.  If  these  attempts  are  not  successful,  on-line  hypotonic 
lysis  of  the  cells  will  be  performed  and  harsher  environmental  conditions  will  be  applied. 

VSOM  perturbations  to  dislodge  cytoplasmic  membrane-associated  AS-ODN  will  consist  of 
buffer  rinses  at  nonphysiogical  pH  or  rinses  with  buffer  containing  trypsin.  Once  again,  quantitative 
data  on  the  magnitude  of  perturbation  required  to  free  the  fluorescently  labeled  AS-ODN  wiU  be 
recorded  on  a.  cell-by-cell  basis  and  will  be  correlated  with  the  chemical  structure  of  the  current 
AS-ODN. 


5.4.2  Physiological  and  fluorescence  assays  for  apoptosis  in  living  cells 

The  physiological  affects  of  the  AS-ODNs  (both  fluorescent  and  nonfluorescent)  can  be  assessed  in 
several  ways.  In  most  cases,  we  will  apply  an  apoptosis-inducing  extracellular  stress  such  as  UV 
light,  anticancer  drugs,  or  calciium  ionophores.  If  the  AS-OND  successfully  reduces  the  amount  of 
bcl-2  protein  in  BT-474  cells,  we  will  be  able  to  detect  a  decrease  in  bcl-2  protein  and  a  decrease  in 
these  cell’s  ability  to  resist  apoptosis.'  The  most  direct  method  is  to  fix  the  cells  after  the  experiment, 
perform  indirect  immunofluorescence  using  an  antibody  for  the  bcl-2  protein,  then  return  to  the 
same  field  of  cells  to  quantify  the  amount  of  bcl-2  protein,  which  should  be  proportional  to  the 
observed  fluorescence  signal  on  a  cell-by-cell  basis. 

Many  other  markers  for  apoptosis  exist,  both  morphological  and  fluorescent.  For  example, 
one  of  the  channels  we  collect  during  VSOM  experiments  is  the  transmitted  light  channel,  where 
membrane  blebbing  can  be  observed.  There  are  also  a  variety  of  antibodies  to  proteins  such  as 
Annexin  V,  and  also  DNA  stains  which  reveal  the  nuclear  fragmentation  characteristic  of  apoptosis. 
In  addition,  we  regularly  use  cytoplasmic  membrane  integrity  (cell  viability)  stains  such  as  calcein- 
AM  in  one  of  the  fluorescence  channels  during  VSOM  experiments.  Also  of  interest  to  us  is  the 
fact  that  loss  of  the  mitochondrial  membrane  potential  occurs  early  in  apoptosis,  and  we  would  be 
able  to  detect  such  changes  in  the  mitochondrial  membrane  potential  bu  using  TMRE,  as  described 
above. 
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5.4.3  AS-ODN  Delivery  techniques 

AS-ODNs  will  be  delivered  as  naked  compound,  or  encapsulated  in  liposomes,  or  they  will  be 

injected  using  a  microcapillary  positioner  we  used  in  our  DNAmicrodissection  studies. 

t 
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ADB282253 

ADB282115 

ADB263413 

ADB269109 

ADB282106 

ADB262514 

ADB282264 

ADB256789 

ADB251569 

ADB258878 

ADB282275 

ADB270822 

ADB282207 

ADB257105 

ADB281673 

ADB254429 

ADB282110 

ADB262549 

ADB268358 

ADB257359 

ADB265810 

ADB282111 

ADB273020 

ADB282185 

ADB266340 

ADB262490 

ADB266385 

ADB282181 

ADB262451 

ADB266306 

ADB260298 

ADB269253 

ADB282119 

ADB261755 

ADB257398 

ADB267683 

ADB282231 

ADB234475 

ADB247704 

ADB258112 

ADB267627 


ADB282838 

ADB233092 

ADB26392? 

ADB282182 

ADB257136 

ADB282227 

ADB282177 

ADB263548 

ADB246535 

ADB282826 

ADB282127 

ADB271165 

ADB282112 

ADB255775 

ADB265599 

ADB282098 

ADB232738 

ADB243196 

ADB257445 

ADB267547 

ADB277556 

ADB239320 

ADB253648 

ADB282171 

ADB233883 

ADB257696 

ADB232089 

ADB240398 

ADB261087 

ADB249593 

ADB264542 

ADB282216 

ADB261617 

ADB269116 


